Response headers

Every response carries a set of X- headers describing the request behind it: its token, whether it came from cache and how stale that cache was, how long we took, and what's left of your allowance. They're the metadata channel — the body stays exactly what you asked for.

Half the endpoints answer in plain text — /markdown, /html, /screenshot and /summarize hand you a body you can pipe straight into a file. Wrapping that in a JSON envelope to carry metadata would break it, so the metadata rides in headers instead. The same headers appear on the JSON endpoints, so one parser reads them everywhere.

Reference

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
X-Result-Token: 0Zx3RkP9…9aQ
X-Cache: hit
X-Cache-Age: 5400
X-Processing-Time-Ms: 12
X-Quota-Category: ai
X-Quota-Limit: 50
X-Quota-Remaining: 43
X-Quota-Reset: 2026-08-31T23:59:59Z

# The page, as Markdown
…

On the webhook

When an async result is delivered by webhook, that POST carries the same facts in its body under a meta object — an accept cannot carry facts about work it hasn't done yet, and you never see headers on a request we make to you. It's an additive field: every existing key keeps its name, type and meaning. A result you fetch with GET /result/:token instead carries them as headers, as any response of ours does.

"meta": {
  "cache": "hit",
  "cache_age": 5400,
  "processing_time_ms": 12,
  "quota": {
    "category": "ai",
    "limit": 50,
    "remaining": 43,
    "resets_at": "2026-08-31T23:59:59Z"
  }
}

Same values, JSON-typed: integers stay integers, and a fact we don't have is an explicit null rather than an absent key — a header consumer checks presence, a webhook consumer parses one shape. The one exception is an unlimited plan, where limit and remaining read "unlimited" exactly as the headers do, so you branch the same way on either channel.

Which responses carry them

• Sync responses — cache hits and cache misses alike.
• Async accepts ({ "status": "accepted" }), where they tell you the token, whether the result was already cached — and, when it was, how long that took, since a cache hit is a finished request already.
• Every GET /result/:token response, including 202 while it's still running and 410 once it's expired.
• Failures with a record behind them: a 422 analysis failure, a 504 sync timeout, and a 429 quota_exceeded (which carries the quota headers).

The exceptions are responses with no request behind them: 401, a 422 for a malformed url or max_age, a 404 for an unknown token, and the 429 rate_limited thrown before the request reaches the endpoint.

The result token

Every request — sync or async, cached or not — gets a token, and X-Result-Token is where you read it. Keep it if you might want the result again: fetching it with GET /result/:token is free, doesn't re-run the analysis, and works for the full 30 day retention window. Async responses also repeat it in the body as token.

Cache status and age

A cache hit is free and instant, so it's worth knowing when you got one. X-Cache tells you which happened:

• hit — served from the result store. No analysis ran, and no quota was spent.
• miss — the analysis ran for this request (or, on an async accept, is about to).
• partial — a /scrape only: some operations were cached and others weren't. The per-operation cached flag in the body says which.

When something was served from cache, X-Cache-Age gives its age in seconds — the time since that result was computed, not since it was last requested. It's always at or below the max_age you asked for, so it tells you how much freshness headroom you actually have. On a partial scrape it's the oldest operation's age: nothing in the body is staler than that.

# Ask for an hour-old result, then see what you got
curl -sD - -o /dev/null -X POST https://urlpipe.dev/markdown \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "sync": true, "max_age": "1 hour"}' \
  | grep -i '^x-cache'

# x-cache: hit
# x-cache-age: 2143      ← 36 minutes old, well inside the hour

Processing time

X-Processing-Time-Ms is your total processing time: from the moment we accept the request to the moment we finish it, in milliseconds. One number, measured the same way for every request — sync, async with a webhook, async without one.

Queueing and analysis are both in it. If a request waited behind others for a free slot against the same site, that wait is yours too and it is in here — there is no separate queue-time or work-time figure to reconcile.

What is not in it is our webhook delivery. How long our delivery queue took to reach your endpoint, and how many times we had to retry, say nothing about how long your analysis took — so a retried delivery reports the same number as the first attempt, and a request you collect from GET /result/:token reports the same number as one we delivered.

It's the number to compare against your own client-side timing — the difference is network. And it's absent until there is something to report: an async accept on a cache miss has no finished work behind it yet, and a 504 means the analysis is still running. It arrives with the finished result, either way: in the webhook's meta object, or on the GET /result/:token response that collects it.

Quota remaining

One pair of numbers answers "how much have I got left": X-Quota-Limit and X-Quota-Remaining. AI and web operations are metered separately, so X-Quota-Category names which of the two allowances the numbers describe — ai for a /markdown call, web for an /html one. It's the same word the 429 quota_exceeded body uses.

A /scrape can draw on both allowances at once. It reports the tighter of the two — the one that will refuse you first — so a single pair of numbers is always the one worth acting on.

The count is taken after your request, so X-Quota-Remaining is what you have left now — not what you had before the call. Cache hits are free, so a hit leaves it unchanged. On an unlimited plan both values read unlimited rather than a number.

const res = await fetch("https://urlpipe.dev/markdown", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ url: "https://example.com", sync: true }),
})

const remaining = res.headers.get("X-Quota-Remaining")
if (remaining !== "unlimited" && Number(remaining) < 25) {
  // Raise max_age to lean on the cache, or slow the crawl down.
  const category = res.headers.get("X-Quota-Category")
  console.warn(`Only ${remaining} ${category} calls left until ${res.headers.get("X-Quota-Reset")}`)
}