Response headers

Every response carries a set of X- headers describing the request behind it: its token, whether it came from cache and how stale that cache was, how long we took, what it cost you in credits, and what's left of your allowance. They're the metadata channel — the body stays exactly what you asked for.

Half the endpoints answer in plain text — /markdown, /html, /screenshot and /summarize hand you a body you can pipe straight into a file. Wrapping that in a JSON envelope to carry metadata would break it, so the metadata rides in headers instead. The same headers appear on the JSON endpoints, so one parser reads them everywhere.

Reference

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
X-Result-Token: 0Zx3RkP9…9aQ
X-Cache: hit
X-Cache-Age: 5400
X-Processing-Time-Ms: 12
X-Quota-Cost: 0
X-Quota-Limit: 1000
X-Quota-Remaining: 943
X-Quota-Overage: 0
X-Quota-Reset: 2026-08-31T23:59:59Z
X-Concurrency-Limit: 1

# The page, as Markdown
…

On the webhook

When an async result is delivered by webhook, that POST carries the same facts in its body under a meta object — an accept cannot carry facts about work it hasn't done yet, and you never see headers on a request we make to you. It's an additive field: every existing key keeps its name, type and meaning. A result you fetch with GET /result/:token instead carries them as headers, as any response of ours does.

"meta": {
  "cache": "hit",
  "cache_age": 5400,
  "processing_time_ms": 12,
  "quota": {
    "cost": 0,
    "limit": 1000,
    "remaining": 943,
    "overage": 0,
    "resets_at": "2026-08-31T23:59:59Z",
    "concurrency_limit": 1
  }
}

Same values, JSON-typed: integers stay integers, and a fact we don't have is an explicit null rather than an absent key — a header consumer checks presence, a webhook consumer parses one shape. The one exception is an unlimited plan, where limit, remaining and concurrency_limit read "unlimited" exactly as the headers do, so you branch the same way on either channel.

Which responses carry them

• Sync responses — cache hits and cache misses alike.
• Async accepts ({ "status": "accepted" }), where they tell you the token, whether the result was already cached — and, when it was, how long that took, since a cache hit is a finished request already.
• Every GET /result/:token response, including 202 while it's still running and 410 once it's expired.
• Failures with a record behind them: a 422 analysis failure, a 504 sync timeout, and a 429 quota_exceeded (which carries the quota headers).

The exceptions are responses with no request behind them: 401, a 422 for a malformed url or max_age, a 404 for an unknown token, and the 429 rate_limited thrown before the request reaches the endpoint.

The result token

Every request — sync or async, cached or not — gets a token, and X-Result-Token is where you read it. Keep it if you might want the result again: fetching it with GET /result/:token is free, doesn't re-run the analysis, and works for the full 30 day retention window. Async responses also repeat it in the body as token.

Cache status and age

A cache hit is free and instant, so it's worth knowing when you got one. X-Cache tells you which happened:

• hit — served from the result store. No analysis ran, and no credits were spent.
• miss — the analysis ran for this request (or, on an async accept, is about to).
• partial — a /scrape only: some operations were cached and others weren't. The per-operation cached flag in the body says which.

When something was served from cache, X-Cache-Age gives its age in seconds — the time since that result was computed, not since it was last requested. It's always at or below the max_age you asked for, so it tells you how much freshness headroom you actually have. On a partial scrape it's the oldest operation's age: nothing in the body is staler than that.

# Ask for an hour-old result, then see what you got
curl -sD - -o /dev/null -X POST https://urlpipe.dev/markdown \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "sync": true, "max_age": "1 hour"}' \
  | grep -i '^x-cache'

# x-cache: hit
# x-cache-age: 2143      ← 36 minutes old, well inside the hour

Processing time

X-Processing-Time-Ms is your total processing time: from the moment we accept the request to the moment we finish it, in milliseconds. One number, measured the same way for every request — sync, async with a webhook, async without one.

Queueing and analysis are both in it. If a request waited behind others for a free slot against the same site, that wait is yours too and it is in here — there is no separate queue-time or work-time figure to reconcile.

What is not in it is our webhook delivery. How long our delivery queue took to reach your endpoint, and how many times we had to retry, say nothing about how long your analysis took — so a retried delivery reports the same number as the first attempt, and a request you collect from GET /result/:token reports the same number as one we delivered.

It's the number to compare against your own client-side timing — the difference is network. And it's absent until there is something to report: an async accept on a cache miss has no finished work behind it yet, and a 504 means the analysis is still running. It arrives with the finished result, either way: in the webhook's meta object, or on the GET /result/:token response that collects it.

What it cost, and what is left

Four numbers, on every response. X-Quota-Cost is what this request spent, in credits — 17 for a /summarize call, 1 for an /html one, 0 for a cache hit or a failed analysis. X-Quota-Limit and X-Quota-Remaining answer "how much of my plan have I got left".

A /scrape reports the sum of what its operations cost, in the same header — there is one allowance, so however many operations a request ran, it has one price. A request that asked for a residential exit has that in the same number too, counted once per page fetch.

The count is taken after your request, so X-Quota-Remaining is what you have left now — not what you had before the call. Cache hits are free, so a hit leaves it unchanged. On an unlimited plan both values read unlimited rather than a number.

Running out of that allowance is not the end of the month on a paid plan — it is not the end of anything. X-Quota-Overage counts the credits spent past it: 0 until the allowance is gone, and after that the number that gets billed when the month closes. There is no companion header for how much further you can go, because on a plan with a card on file you are never cut off for credits. On the Free plan there is no overage and the allowance is the wall. See overage.

Sizing a worker pool

X-Concurrency-Limit is how many requests your plan can run in parallel. It is a property of the plan rather than a measurement, so it costs us nothing to send and it is the same on every response — which makes it the right number to size a worker pool from, instead of discovering the limit by being refused.

There is deliberately no companion header counting what is in flight right now. It would be a database query on every response for a number that is already out of date by the time you read it; a client that tracks its own outstanding requests knows sooner and more accurately. The concurrency_limit 429 carries the live count when it actually matters.

const res = await fetch("https://urlpipe.dev/markdown", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ url: "https://example.com", sync: true }),
})

// What this call cost, and what is left afterwards.
const cost = Number(res.headers.get("X-Quota-Cost"))
const remaining = res.headers.get("X-Quota-Remaining")
const overage = Number(res.headers.get("X-Quota-Overage"))

// Past the included allowance: still being served — always — now at
// $1.50 per 1,000 credits. Worth logging the first time it happens.
if (overage > 0) {
  console.warn(`${overage} credits into overage this month`)
}

// Compare against the cost of the call you are about to make, not a
// flat number: 17 credits is one more /summarize, or seventeen /html.
if (remaining !== "unlimited" && Number(remaining) < cost) {
  // Raise max_age to lean on the cache, reach for a cheaper endpoint,
  // or wait for the reset.
  console.warn(`${remaining} credits left until ${res.headers.get("X-Quota-Reset")}`)
}