Changelog · September 24, 2026
page_options: clean up the page before anything is read
Get the page a reader sees, without the consent panel's cookie policy at the top of your Markdown.
By Roger Campos · Last updated: September 2026
TL;DR
Every endpoint that loads a page now takes page_options: block_ads, block_cookie_banners, remove_selectors, wait_for_selector and delay. They apply to the page once it has settled and before anything is read off it, and what they take out is removed from the document, not hidden — so it is gone from the HTML and Markdown too, not just the screenshot.
Free plan, no credit card. 1,000 credits a month.
Waiting and blocking used to belong to the screenshot and hid things with CSS: the pixels changed, the page did not, and a consent panel's policy text went on arriving in the HTML and Markdown. The page_options object applies to the page itself, so /html, /markdown, /meta, /summarize, /keywords, /console and /screenshot all read the same cleaned page.
Measured on a cookie-consent vendor's own site: /markdown went from 27,900 characters with 117 mentions of "cookie" to 8,545 with none. Cookie banners are removed, never answered — clicking "accept" would consent on your behalf.
Request
Example
curl -X POST https://urlpipe.dev/markdown \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "sync": true, "page_options": {"block_ads": true, "block_cookie_banners": true, "remove_selectors": [".newsletter"]}}'/lighthouse refuses page options with invalid_options: an audit measures its own page load, and blocking requests would change the scores. In a /scrape they apply to every operation except the audit.
Endpoints
The endpoints on this page
- Turn any URL into clean Markdown
Paste a link and get the page's main content as tidy Markdown — headings, lists, links and code kept, navigation and cookie banners stripped. The format LLMs and RAG pipelines work best with.
Try it free - Get the rendered HTML of any URL
Paste a link and get the page's HTML after JavaScript has run and redirects have been followed — the DOM a real browser sees, not the empty shell curl returns.
Try it free
FAQ
Frequently asked questions
Are removed elements hidden or deleted?
Does URLpipe click the cookie banner's accept button?
Can I use page_options with /lighthouse?
Try it on the free plan.
Free plan, no card. Confirm your email and your API key is live — you'll be making real requests in minutes.