Skip to main content

Confirm

Are you sure?

URLpipe
Sign inStart free

URLpipe

Changelog · September 24, 2026

Idempotency-Key, and identical requests share one run

Retry after a timeout or a dropped connection without paying for the same page twice.

By · Last updated: September 2026

TL;DR

Two changes so a repeated request is answered once. An identical request that arrives while the first is still running now waits for it and is settled with its answer, with no credits spent. And an Idempotency-Key header makes a retry return the first request — same token, result, charge and webhook — for 24 hours.

Start freeSee the API

Free plan, no credit card. 1,000 credits a month.

Before this, a duplicate that arrived while the first request was still running missed the result store, visited the page again and was charged again. It now waits on the running one: it gets its own token, labels and webhook, but no job, no credits and no parallel-request slot. max_age: 0 never shares, because it asks for a page fetched after the request.

For a retry you control, send an Idempotency-Key header (idempotency_key over MCP). The retry returns the first request's token, result, charge and webhook for 24 hours, in whichever mode the retry asks for. The same key on a different request is refused with idempotency_key_reused.

Request

Example

A retry-safe request
curl -X POST https://urlpipe.dev/markdown \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Idempotency-Key: 5f1c9e2a-order-4411" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "sync": true}'
Retries & duplicatesCaching

Endpoints

The endpoints on this page

/markdown API docs/scrape API docs

FAQ

Frequently asked questions

How long is an Idempotency-Key remembered?
24 hours. Within that window a retry with the same key returns the first request.
What happens if I reuse a key for a different request?
It is refused with idempotency_key_reused, so a key can never return an answer to a question you didn't ask.
Does a duplicate that waits cost credits?
No. It spends no credits and holds no parallel-request slot; it is settled with the first request's answer.

Keep reading

URLpipe vs FirecrawlLangChainWeb pages into a RAG pipelineApify alternativesLlamaIndexWhy AI agents read empty pagesAll of changelog

Try it on the free plan.

Free plan, no card. Confirm your email and your API key is live — you'll be making real requests in minutes.

Start freeEmail the team