Residential exits

Some sites read the address a request came from and serve a server farm less than they serve a person. Add one parameter and URLpipe fetches the page from a home broadband line instead.

{ "url": "https://example.com", "residential": true }

It works on every endpoint, including /scrape, and it changes nothing else about the request: the same response, the same headers, the same sync and async modes. It defaults to false, so nothing you already run is affected.

When it helps

Try without it first. Most pages come back complete on the standard network, and when a site puts a bot check in front of us we work it and retry before you ever hear about it. Reach for a residential exit when that isn't enough:

• The response is The page asked us to complete a bot check before it would load. — a site that will not clear for a datacentre address, however long we wait.
• The page loads but comes back thin: a paywall stub, a consent wall, or a product page with the prices missing.
• You need what a real visitor sees — regional pricing, local inventory, the page a household connection is served rather than the one a bot is.

What it costs

A residential exit adds 25 credits to a request — charged once per page fetch, not once per operation. Residential bandwidth is bought by the gigabyte and a page visit spends a page's worth of it, which is the whole of why it is priced separately from the work done on the page afterwards.

That is the one place a /scrape is cheaper than the same operations called one at a time: it visits the page once and reads every operation off that single visit, so it pays for one exit however many results come back. A /lighthouse runs its own audit on its own engine, so a scrape that includes it fetches twice and pays twice.

The usual rules hold. A result served from cache fetched nothing, so it costs nothing — exit included. A failed analysis costs nothing either, even though the fetch was attempted. And X-Quota-Cost on every response reports the whole figure, exit and operations together.

Residential results are kept apart

The point of asking for a home address is that the page may answer differently, so a residential result is stored under its own key. A residential request is never handed a result fetched from the datacentre, and an ordinary request is never handed a residential one. Repeat the same residential call inside your max_age window and it is a free cache hit like any other.

When no exit is free

Residential capacity is finite and, unlike the rest of the network, it has no stand-in: answering from a datacentre address would be the one thing the request asked us not to do. So when nothing is free the request fails, plainly and for free:

{
  "error": "No residential exit was free to load this page. Retry shortly, or send the same request without residential to use our standard network."
}

Nothing is spent. Retry in a moment, or drop the parameter and take the standard network's answer — which for most pages is the same answer.

Trying it out

Point it at the targets that are giving you trouble and compare. A free API key comes with 1,000 credits a month — enough to fetch 38 pages through a residential exit and find out whether it changes what they send back, before you build anything on it.

The free tools and the in-app Playground always use the standard network, so this is the one part of URLpipe you can only try with a key.