PHP · Code recipe
Convert a web page to Markdown in PHP
Turn any URL into Markdown you can hand to a model, in PHP 8.1+ with the curl extension. Every program on this page runs as it stands — each one was run against a stub of the API before it was published.
By Roger Campos · Last updated: September 2026
TL;DR
To convert a web page to Markdown in PHP, POST its URL to https://urlpipe.dev/markdown with sync set to true; the response body is the Markdown, read with curl_exec with CURLOPT_RETURNTRANSFER. The page is rendered in real Chrome first, then converted by a deterministic walk over the DOM, not a model, for 1 credit a page.
Free plan, no credit card. 1,000 credits a month.
One POST to /markdown renders the page in real Chrome, then walks the rendered DOM and writes the main content as Markdown, with navigation, footers and boilerplate left out. No model is involved, so the same page gives the same Markdown every time, in about 20 ms after the page has loaded.
In PHP the response is plain text, read with curl_exec with CURLOPT_RETURNTRANSFER — there is no JSON envelope to unwrap. Measured on 33 pages, the output covered 87.2% of the text a visitor sees, and 11.0% of it was text the visitor never saw; the guide to Markdown for LLMs has the comparison.
Setup
Before you start
Nothing to install beyond PHP itself: the curl extension ships with it. Check it is loaded, and put your API key in the environment.
php -m | grep curl # prints "curl"
export URLPIPE_API_KEY="your_api_key"
The request
Print a page as Markdown
Without CURLOPT_RETURNTRANSFER, curl_exec() prints the body and returns true — the most common reason a PHP example "returns nothing". curl_exec() returns false only when there was no HTTP answer at all; a 422 is still a string, so read the status too. The body is the Markdown itself, so $body is the whole job.
<?php
function fail(string $message): never
{
fwrite(STDERR, $message . "\n");
exit(1);
}
$ch = curl_init("https://urlpipe.dev/markdown");
curl_setopt_array($ch, [
CURLOPT_POST => true,
// Return the body from curl_exec() instead of printing it.
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => [
"Authorization: Bearer " . getenv("URLPIPE_API_KEY"),
"Content-Type: application/json",
],
CURLOPT_POSTFIELDS => json_encode(["url" => "https://example.com", "sync" => true]),
// A sync call can take up to 60 s; curl would otherwise wait forever.
CURLOPT_TIMEOUT => 90,
]);
$body = curl_exec($ch);
if ($body === false) {
fail("Request failed: " . curl_error($ch));
}
$status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
if ($status !== 200) {
fail("URLpipe answered {$status}: {$body}");
}
// Plain text: pipe it into a file, a chunker or a prompt.
echo $body, "\n";
Run it: php html_to_markdown.php
Async
The async variant: a token, a webhook and a poll
Leave out sync and the answer is a token, straight away. send() uses CURLOPT_CUSTOMREQUEST so one function makes both the POST and the GETs, and array destructuring ([$status, $body] = …) keeps the loop short.
<?php
const API = "https://urlpipe.dev";
function fail(string $message): never
{
fwrite(STDERR, $message . "\n");
exit(1);
}
// One request to the API: returns [status, body].
function send(string $method, string $path, ?array $payload = null): array
{
$ch = curl_init(API . $path);
curl_setopt_array($ch, [
CURLOPT_CUSTOMREQUEST => $method,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => [
"Authorization: Bearer " . getenv("URLPIPE_API_KEY"),
"Content-Type: application/json",
],
CURLOPT_TIMEOUT => 30,
]);
if ($payload !== null) {
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($payload));
}
$body = curl_exec($ch);
if ($body === false) {
fail("Request failed: " . curl_error($ch));
}
return [curl_getinfo($ch, CURLINFO_RESPONSE_CODE), $body];
}
// No "sync": the request is accepted at once and the work carries on without you.
[$status, $body] = send("POST", "/markdown", [
"url" => "https://example.com",
"report_to" => "https://your-app.com/webhooks/urlpipe",
"labels" => ["customer" => "acme"],
]);
if ($status !== 200) {
fail("URLpipe answered {$status}: {$body}");
}
$token = json_decode($body, true)["token"];
echo "Accepted {$token}\n";
// The result is POSTed to report_to when it is ready. Polling by token is the
// other way to collect it: no endpoint needed, and a backup for the webhook.
for ($attempt = 0; $attempt < 60; $attempt++) {
[$status, $body] = send("GET", "/result/{$token}");
if ($status !== 202) { // 202 means still processing
break;
}
sleep(2);
}
if ($status === 202) {
fail("Still processing after two minutes; try the token again later.");
}
if ($status === 422) {
fail("The analysis failed: " . json_decode($body, true)["error"]);
}
if ($status === 410) {
fail("The result is past the 30-day window; send the request again.");
}
if ($status !== 200) {
fail("URLpipe answered {$status}: {$body}");
}
// Plain text: pipe it into a file, a chunker or a prompt.
echo $body, "\n";
Run it: php html_to_markdown_async.php
Errors
Handle errors and retries
json_decode() returns null for a body that is not JSON (a 401 answers in plain text), and ?: [] turns that into an empty array. CURLOPT_HEADERFUNCTION is how curl hands you response headers one line at a time.
<?php
final class URLpipeError extends RuntimeException
{
}
// POST a sync request and return the result body, or throw URLpipeError.
function urlpipe(string $path, array $payload, int $attempts = 5): string
{
for ($attempt = 0; $attempt < $attempts; $attempt++) {
$headers = [];
$ch = curl_init("https://urlpipe.dev{$path}");
curl_setopt_array($ch, [
CURLOPT_POST => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => [
"Authorization: Bearer " . getenv("URLPIPE_API_KEY"),
"Content-Type: application/json",
],
CURLOPT_POSTFIELDS => json_encode($payload + ["sync" => true]),
CURLOPT_TIMEOUT => 90,
// Collect the response headers: Retry-After is the one needed here.
CURLOPT_HEADERFUNCTION => function ($ch, string $line) use (&$headers): int {
[$name, $value] = array_pad(explode(":", $line, 2), 2, "");
$headers[strtolower(trim($name))] = trim($value);
return strlen($line);
},
]);
$body = curl_exec($ch);
if ($body === false) {
throw new URLpipeError("network error: " . curl_error($ch));
}
$status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
if ($status === 200) {
return $body;
}
if ($status === 401) {
throw new URLpipeError("401: the API key is missing or wrong. Check URLPIPE_API_KEY.");
}
$error = json_decode($body, true) ?: [];
$code = $error["error"] ?? "";
$detail = isset($error["message"]) ? "{$code}: {$error["message"]}" : $code;
if ($status === 429 && $code === "rate_limited") {
// Sending too fast: Retry-After says how long the window has left.
sleep((int) ($headers["retry-after"] ?? 1));
} elseif ($status === 429 && $code === "concurrency_limit") {
// Every parallel slot on your plan is busy with your own requests.
sleep(2 ** $attempt);
} elseif ($status === 504) {
// Still running on our side; the token collects it from GET /result/:token.
throw new URLpipeError("504 processing_timeout: collect it later with token {$error["token"]}");
} else {
// 403 email_unverified, 422 (a bad parameter, or a page that would not load),
// 429 quota_exceeded: sending the same request again gets the same answer.
throw new URLpipeError("{$status}: {$detail}");
}
}
throw new URLpipeError("429: still refused after {$attempts} attempts");
}
try {
$body = urlpipe("/markdown", ["url" => "https://example.com"]);
} catch (URLpipeError $e) {
fwrite(STDERR, "URLpipe: {$e->getMessage()}\n");
exit(1);
}
// Plain text: pipe it into a file, a chunker or a prompt.
echo $body, "\n";
Run it: php html_to_markdown_errors.php
Details
What to know about /markdown
- It reads no CSS, so text hidden only by a stylesheet can end up in the Markdown.
- Pages over 10 MB of HTML are refused with
The page is too big to be processed. page_options.remove_selectorsdrops elements before conversion when a site's chrome survives the boilerplate rules.- A repeat request inside
max_age(7 days by default) is served from the cache and costs nothing.
FAQ
Frequently asked questions
Does the Markdown conversion use an LLM?
Does it work on pages built with JavaScript?
How do I keep the Markdown for later?
Do I need an SDK to call URLpipe from PHP?
Get a key and run it.
Free plan, no card. Paste your key into URLPIPE_API_KEY and every program on this page runs as it is.