Web scraping API
A web scraping API for the URLs you already have
One call renders the page in real Chrome and hands back its Markdown, metadata and a screenshot — or any mix of 11 results — from a single visit.
By Roger Campos · Last updated: October 2026
TL;DR
POST a URL and a list of operations to /scrape and get every result from one page visit: the page is rendered in real Chrome, and its Markdown, metadata, screenshot, PDF, rendered HTML, console output, technologies, summary, keywords or structured data come back in one JSON response. Each operation costs what its own endpoint costs — Markdown, metadata and a screenshot are 3 credits together — and cache hits and failed requests are free.
Free plan, no credit card. 1,000 credits a month.
Try it now — no signup
Your result will appear here.
Enter a URL above, or pick one of the example pages.
The request
One POST, one bearer token
curl -X POST https://urlpipe.dev/scrape \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "operations": ["markdown","meta","screenshot"], "sync": true}'import requests
res = requests.post(
"https://urlpipe.dev/scrape",
headers={"Authorization": "Bearer YOUR_API_KEY"},
json={"url": "https://example.com", "operations": ["markdown", "meta", "screenshot"], "sync": True},
)const res = await fetch("https://urlpipe.dev/scrape", {
method: "POST",
headers: {
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
body: JSON.stringify({ url: "https://example.com", operations: ["markdown","meta","screenshot"], sync: true }),
})$ch = curl_init("https://urlpipe.dev/scrape");
curl_setopt_array($ch, [
CURLOPT_POST => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => [
"Authorization: Bearer YOUR_API_KEY",
"Content-Type: application/json",
],
CURLOPT_POSTFIELDS => json_encode(["url" => "https://example.com", "operations" => ["markdown", "meta", "screenshot"], "sync" => true]),
]);
$response = curl_exec($ch);{
"url": "https://example.com",
"final_url": "https://example.com/",
"operations": {
"markdown": {
"success": true,
"result": "# Example Domain\n\nThis domain is for use in…",
"cached": false
},
"meta": {
"success": true,
"result": { "title": "Example Domain", "description": "…", "language": "en" },
"cached": true
},
"screenshot": {
"success": true,
"result": "iVBORw0KGgoAAAANSUhEUg…",
"cached": false
}
}
}One visit
What one call returns
Name the results you want in operations. The page loads once, and every result is read off that visit: the rendered HTML, the screenshot and the console come from the capture, and the Markdown, metadata and AI results are derived from the HTML. Loading the page is the slow part of any scrape, so three results cost one wait instead of three.
| Operation | What it returns | Credits |
|---|---|---|
| markdown | The page's main content as Markdown | 1 |
| html | The rendered HTML, after JavaScript | 1 |
| meta | Title, description, share image, authors, dates, feeds | 1 |
| screenshot | The page as PNG, JPEG or WebP | 1 |
| The page as a PDF | 1 | |
| console | Errors, warnings and uncaught exceptions | 1 |
| technologies | CMS, frameworks, analytics, CDN, with versions | 1 |
| lighthouse | Lighthouse scores and Core Web Vitals lab metrics | 2 |
| keywords | Keywords from the page, by an AI model | 15 |
| summarize | A summary of the page, by an AI model | 17 |
| extract | Data in the fields you define, by an AI model | 24 |
Every result is identical to what its own endpoint returns and is stored under the same key, so a scrape can be answered from a /markdown call you made yesterday, and a /meta call tomorrow from today's scrape. Operations fail one at a time: a failed one leaves its siblings intact and costs nothing. How the single visit works.
Rendering
JavaScript sites come back complete
Every page loads in real headless Chrome and its scripts run before anything is read, so a React, Vue or Angular app returns its content instead of an empty <div id="root">. There is no rendering flag to remember and no surcharge for it. page_options removes cookie banners, ads and popups before the capture, and applies to every operation of the scrape. Rendered and raw HTML, compared.
Responsibility
robots.txt, followed by default
Each project follows the robots.txt of the sites it fetches unless you switch it off in the project's settings. A page robots.txt disallows is never requested, and the call fails with an error that says why, at no cost. Switching it off is your call, and so is the right to fetch what you fetch. How the rules are read.
Residential
Residential IPs for sites that block data centers
Some sites show a data center address a block page, a captcha or a thinner page than they show a person. Add residential: true and the page is fetched from a home broadband connection instead, for 25 credits on top of the operations. On /scrape that is charged once per visit, so five results from one residential visit pay it once, not five times. How residential fetches work.
Billing
A bill that matches the work
- Cache hits are free. Results are stored and reused while they are fresher than
max_age, so scraping a page you already scraped costs nothing. Caching. - Failures are free. A page that doesn't load, a bot check that isn't cleared, an operation that fails: none of them spend credits.
- Never cut off. On a paid plan, a busy month doesn't stop your pipeline: past the allowance, extra credits are billed at the overage rate.
- One request against your parallel limit. A scrape counts once however many operations it runs, because it is one page visit.
Limits
What it does not do
- No crawling. It scrapes the URLs you send and doesn't discover or follow links. For whole-site crawls, look at Apify or Crawl4AI.
- No interaction. It doesn't click, type or fill forms; it reads the page as it loads — signed in, for a page behind a login, when the request names an auth profile.
- Public addresses only. Private, loopback and intranet addresses are refused.
Pricing
What it costs
| Call | Credits | Free (calls/mo) | Starter (calls/mo) | Pro (calls/mo) | Scale (calls/mo) |
|---|---|---|---|---|---|
| /markdown | 1 credit | 1,000 | 20,000 | 55,000 | 175,000 |
| /meta | 1 credit | 1,000 | 20,000 | 55,000 | 175,000 |
| /screenshot | 1 credit | 1,000 | 20,000 | 55,000 | 175,000 |
Cache hits and failed requests cost nothing. Paid plans are never cut off: past the allowance, extra credits are $1.50 per 1,000 credits. See every plan.
Endpoints
The endpoints on this page
- Everything about a URL, from one visit
Paste a link and get a screenshot of the page, its title, description and share image, the main content as Markdown and the console errors it threw — all from the same page load.
Try it free - Turn any URL into clean Markdown
Paste a link and get the page's main content as tidy Markdown — headings, lists, links and code kept, navigation and cookie banners stripped. The format LLMs and RAG pipelines work best with.
Try it free - Check a page's Open Graph tags and metadata
Paste a link and see everything the page says about itself — title, description, share image, authors, dates, language, icons and feeds — as one structured object.
Try it free - Screenshot any website from its URL
Paste a link and get a sharp PNG of the page as it opens — JavaScript executed, exactly as a real browser would draw it. Great for previews, thumbnails and visual QA.
Try it free
FAQ
Frequently asked questions
Yes. Every page loads in headless Chrome with its scripts running before anything is read, at no extra cost.
The sum of its operations: Markdown, metadata and a screenshot are 3 credits together. Cached operations and failed requests are free, and a residential exit is charged once per visit.
No. It scrapes the URLs you send it, one page per request. For whole-site crawls, a crawler such as Apify or Crawl4AI fits better.
Yes. Sign in once in a browser, upload its saved state (Playwright's storageState) as an auth profile, and name it as auth_profile on the request: the page is loaded signed in, with nothing to log in again, at no extra cost. Whatever the site renews during a visit is kept, so the session stays alive while you use it.
Yes, by default. Each project follows robots.txt unless an admin switches it off in the project's settings.
Yes. Each operation is produced and stored exactly as its own endpoint would, so either can be served from the other's cache.
Make your first request in five minutes.
Free plan, no card. Confirm your email and your API key is live — you'll be making real requests in minutes.