Changelog
What's new in URLpipe
Notable changes to the API, the MCP server and the plans, newest first.
- Each project chooses whether to follow robots.txtA project setting, on by default: a page the site's robots.txt disallows is not fetched, and the refusal costs nothing.Read
- Idempotency-Key, and identical requests share one runA retried request returns the first one's result and charge; a duplicate that arrives mid-run waits for it for free.Read
- Labels: tag requests with your own idsAttach up to 16 key/value labels to any request, get them back everywhere, and see usage broken down by label.Read
- page_options: clean up the page before anything is readBlock ads, remove cookie banners and your own selectors, or wait for an element — for every result read off the page.Read
- screenshot_options: viewport, format, element, dark modePNG, JPEG or WebP, viewport and scale, one element by selector, dark mode, hidden selectors and injected CSS — at one credit.Read
- Paid plans, VAT, and paid plans are never cut offStarter, Pro and Scale on sale through Stripe; past the allowance, paid plans carry on and extra credits are billed.Read
- Screenshots capture the whole pageFull page by default, with lazy-loaded images loaded first, up to a 16,384 px tall capture.Read
- Every screenshot has a link that needs no API keyX-Result-Url gives a signed link to the image, valid for as long as the result is stored — drop it straight into an image tag.Read
- Keep AI processing in the EU, per organizationAn organization setting that keeps /meta, /summarize and /keywords inside the European Union, at no extra charge.Read
- API keys are stored hashed, and shown onceProject keys are stored as a digest, displayed only when minted, and can be rotated with a 24-hour overlap.Read
- The URLpipe API as an MCP serverPoint Claude, Cursor or any MCP client at urlpipe.dev/mcp: 14 tools, the same arguments, results and credits as the API.Read
- Fetch a page from a residential addressresidential: true routes the page visit through a home internet connection, for sites that answer datacentre traffic differently.Read
- Parallel requests per planEach plan runs a set number of requests at once — 1, 3, 8 or 20 — and says so on every response.Read
- One credit meter, priced by what each call costsThe two separate AI and web allowances became one monthly credit balance, with each endpoint priced by what it costs to run.Read
- /markdown converts without a language modelA DOM walk replaces the model: better coverage, half the stray text, about 20 ms a page, and no model cost.Read
- Close your account or delete an organization yourselfAccount settings closes an account; Organization settings deletes an organization and everything in it.Read
- A site that can't keep up is paced, not failedWhen a target site slows down or answers 429/5xx, requests to it wait and retry instead of failing.Read
- Filter requests and webhooks in the dashboardNarrow History by duration, URL and outcome, and the Webhooks table by endpoint and delivery time.Read
- Send a failed webhook again by handA Retry now button on any failed delivery, at any age, so a fixed endpoint can still receive the result.Read
- Async requests no longer need a webhookreport_to is optional: collect async results with GET /result/:token, or set a default webhook per project.Read
- Signed webhooksEvery delivery can carry an HMAC-SHA256 signature over the timestamp and raw body, with a 24-hour overlap on rotation.Read
- Cache, timing and quota on every responseX-Result-Token, X-Cache, X-Cache-Age, X-Processing-Time-Ms and the quota headers on every response, and a meta object in webhooks.Read
- POST /scrape: several results off one page visitAsk for any combination of operations on one URL; the page loads once and every result comes back together.Read
- Failed requests are never chargedA timeout, a DNS error or the target's own 4xx/5xx spends nothing — and an error page is no longer returned as a result.Read
- Free tools: try every endpoint without signing upA no-signup page for each endpoint that runs it live on a URL you choose and shows the curl that reproduces it.Read
- GET /result/:tokenFetch any result by its token for 30 days — the way to collect an async result, or a sync one that timed out.Read
- Per-project rate limits60 requests a minute and 15 per 10 seconds per project key, with a 429 and Retry-After when you go over.Read
- Sign in with Google or GitHubCreate an account or sign in with a Google or GitHub account instead of a password.Read
- Public API documentation at /docsA docs site with a quickstart, the core concepts and a page per endpoint, open without an account.Read
- Request caching with max_ageRepeat requests are served from stored results inside a freshness window you choose — free.Read
Turn any URL into clean data in minutes.
Free plan, no card. Confirm your email and your API key is live — you'll be making real requests in minutes.