Skip to main content

Confirm

Are you sure?

PHP · Code recipe

Convert a web page to Markdown in PHP

Turn any URL into Markdown you can hand to a model, in PHP 8.1+ with the curl extension. Every program on this page runs as it stands — each one was run against a stub of the API before it was published.

By · Last updated: September 2026

TL;DR

To convert a web page to Markdown in PHP, POST its URL to https://urlpipe.dev/markdown with sync set to true; the response body is the Markdown, read with curl_exec with CURLOPT_RETURNTRANSFER. The page is rendered in real Chrome first, then converted by a deterministic walk over the DOM, not a model, for 1 credit a page.

Free plan, no credit card. 1,000 credits a month.

One POST to /markdown renders the page in real Chrome, then walks the rendered DOM and writes the main content as Markdown, with navigation, footers and boilerplate left out. No model is involved, so the same page gives the same Markdown every time, in about 20 ms after the page has loaded.

In PHP the response is plain text, read with curl_exec with CURLOPT_RETURNTRANSFER — there is no JSON envelope to unwrap. Measured on 33 pages, the output covered 87.2% of the text a visitor sees, and 11.0% of it was text the visitor never saw; the guide to Markdown for LLMs has the comparison.

Setup

Before you start

Nothing to install beyond PHP itself: the curl extension ships with it. Check it is loaded, and put your API key in the environment.

Terminal
php -m | grep curl   # prints "curl"
export URLPIPE_API_KEY="your_api_key"

The request

Print a page as Markdown

Without CURLOPT_RETURNTRANSFER, curl_exec() prints the body and returns true — the most common reason a PHP example "returns nothing". curl_exec() returns false only when there was no HTTP answer at all; a 422 is still a string, so read the status too. The body is the Markdown itself, so $body is the whole job.

html_to_markdown.php
<?php

function fail(string $message): never
{
    fwrite(STDERR, $message . "\n");
    exit(1);
}

$ch = curl_init("https://urlpipe.dev/markdown");
curl_setopt_array($ch, [
    CURLOPT_POST => true,
    // Return the body from curl_exec() instead of printing it.
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_HTTPHEADER => [
        "Authorization: Bearer " . getenv("URLPIPE_API_KEY"),
        "Content-Type: application/json",
    ],
    CURLOPT_POSTFIELDS => json_encode(["url" => "https://example.com", "sync" => true]),
    // A sync call can take up to 60 s; curl would otherwise wait forever.
    CURLOPT_TIMEOUT => 90,
]);
$body = curl_exec($ch);
if ($body === false) {
    fail("Request failed: " . curl_error($ch));
}
$status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
if ($status !== 200) {
    fail("URLpipe answered {$status}: {$body}");
}

// Plain text: pipe it into a file, a chunker or a prompt.
echo $body, "\n";

Run it: php html_to_markdown.php

Async

The async variant: a token, a webhook and a poll

Leave out sync and the answer is a token, straight away. send() uses CURLOPT_CUSTOMREQUEST so one function makes both the POST and the GETs, and array destructuring ([$status, $body] = …) keeps the loop short.

html_to_markdown_async.php
<?php

const API = "https://urlpipe.dev";

function fail(string $message): never
{
    fwrite(STDERR, $message . "\n");
    exit(1);
}

// One request to the API: returns [status, body].
function send(string $method, string $path, ?array $payload = null): array
{
    $ch = curl_init(API . $path);
    curl_setopt_array($ch, [
        CURLOPT_CUSTOMREQUEST => $method,
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_HTTPHEADER => [
            "Authorization: Bearer " . getenv("URLPIPE_API_KEY"),
            "Content-Type: application/json",
        ],
        CURLOPT_TIMEOUT => 30,
    ]);
    if ($payload !== null) {
        curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($payload));
    }
    $body = curl_exec($ch);
    if ($body === false) {
        fail("Request failed: " . curl_error($ch));
    }
    return [curl_getinfo($ch, CURLINFO_RESPONSE_CODE), $body];
}

// No "sync": the request is accepted at once and the work carries on without you.
[$status, $body] = send("POST", "/markdown", [
    "url" => "https://example.com",
    "report_to" => "https://your-app.com/webhooks/urlpipe",
    "labels" => ["customer" => "acme"],
]);
if ($status !== 200) {
    fail("URLpipe answered {$status}: {$body}");
}
$token = json_decode($body, true)["token"];
echo "Accepted {$token}\n";

// The result is POSTed to report_to when it is ready. Polling by token is the
// other way to collect it: no endpoint needed, and a backup for the webhook.
for ($attempt = 0; $attempt < 60; $attempt++) {
    [$status, $body] = send("GET", "/result/{$token}");
    if ($status !== 202) { // 202 means still processing
        break;
    }
    sleep(2);
}
if ($status === 202) {
    fail("Still processing after two minutes; try the token again later.");
}
if ($status === 422) {
    fail("The analysis failed: " . json_decode($body, true)["error"]);
}
if ($status === 410) {
    fail("The result is past the 30-day window; send the request again.");
}
if ($status !== 200) {
    fail("URLpipe answered {$status}: {$body}");
}

// Plain text: pipe it into a file, a chunker or a prompt.
echo $body, "\n";

Run it: php html_to_markdown_async.php

Errors

Handle errors and retries

json_decode() returns null for a body that is not JSON (a 401 answers in plain text), and ?: [] turns that into an empty array. CURLOPT_HEADERFUNCTION is how curl hands you response headers one line at a time.

html_to_markdown_errors.php
<?php

final class URLpipeError extends RuntimeException
{
}

// POST a sync request and return the result body, or throw URLpipeError.
function urlpipe(string $path, array $payload, int $attempts = 5): string
{
    for ($attempt = 0; $attempt < $attempts; $attempt++) {
        $headers = [];
        $ch = curl_init("https://urlpipe.dev{$path}");
        curl_setopt_array($ch, [
            CURLOPT_POST => true,
            CURLOPT_RETURNTRANSFER => true,
            CURLOPT_HTTPHEADER => [
                "Authorization: Bearer " . getenv("URLPIPE_API_KEY"),
                "Content-Type: application/json",
            ],
            CURLOPT_POSTFIELDS => json_encode($payload + ["sync" => true]),
            CURLOPT_TIMEOUT => 90,
            // Collect the response headers: Retry-After is the one needed here.
            CURLOPT_HEADERFUNCTION => function ($ch, string $line) use (&$headers): int {
                [$name, $value] = array_pad(explode(":", $line, 2), 2, "");
                $headers[strtolower(trim($name))] = trim($value);
                return strlen($line);
            },
        ]);
        $body = curl_exec($ch);
        if ($body === false) {
            throw new URLpipeError("network error: " . curl_error($ch));
        }
        $status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
        if ($status === 200) {
            return $body;
        }
        if ($status === 401) {
            throw new URLpipeError("401: the API key is missing or wrong. Check URLPIPE_API_KEY.");
        }

        $error = json_decode($body, true) ?: [];
        $code = $error["error"] ?? "";
        $detail = isset($error["message"]) ? "{$code}: {$error["message"]}" : $code;

        if ($status === 429 && $code === "rate_limited") {
            // Sending too fast: Retry-After says how long the window has left.
            sleep((int) ($headers["retry-after"] ?? 1));
        } elseif ($status === 429 && $code === "concurrency_limit") {
            // Every parallel slot on your plan is busy with your own requests.
            sleep(2 ** $attempt);
        } elseif ($status === 504) {
            // Still running on our side; the token collects it from GET /result/:token.
            throw new URLpipeError("504 processing_timeout: collect it later with token {$error["token"]}");
        } else {
            // 403 email_unverified, 422 (a bad parameter, or a page that would not load),
            // 429 quota_exceeded: sending the same request again gets the same answer.
            throw new URLpipeError("{$status}: {$detail}");
        }
    }
    throw new URLpipeError("429: still refused after {$attempts} attempts");
}

try {
    $body = urlpipe("/markdown", ["url" => "https://example.com"]);
} catch (URLpipeError $e) {
    fwrite(STDERR, "URLpipe: {$e->getMessage()}\n");
    exit(1);
}

// Plain text: pipe it into a file, a chunker or a prompt.
echo $body, "\n";

Run it: php html_to_markdown_errors.php

Details

What to know about /markdown

  • It reads no CSS, so text hidden only by a stylesheet can end up in the Markdown.
  • Pages over 10 MB of HTML are refused with The page is too big to be processed.
  • page_options.remove_selectors drops elements before conversion when a site's chrome survives the boilerplate rules.
  • A repeat request inside max_age (7 days by default) is served from the cache and costs nothing.

Other languages

Convert a web page to Markdown in another language

More PHP: every PHP recipe

FAQ

Frequently asked questions

Does the Markdown conversion use an LLM?
No. It is a deterministic walk over the rendered DOM, so it is fast, costs 1 credit a page, and gives the same output for the same page. Only /meta, /summarize and /keywords call a model.
Does it work on pages built with JavaScript?
Yes. The page is loaded in real Chrome and its scripts run before the DOM is read, so React, Vue and other client-rendered pages convert like static ones.
How do I keep the Markdown for later?
Write the body to a .md file, or rely on the cache: the same URL within max_age (7 days by default) is answered from storage for free, and any result can be fetched again by its token for 30 days.
Do I need an SDK to call URLpipe from PHP?
No Composer package: the curl extension is part of PHP, and the API is one POST per job.

Get a key and run it.

Free plan, no card. Paste your key into URLPIPE_API_KEY and every program on this page runs as it is.