Page options

Wait for the page you actually want, and take out what you don't — ads, cookie banners, a newsletter pop-up — before anything is read off it. What goes is gone from every result: the HTML, the Markdown, the metadata, the summary, the keywords, the console and the screenshot.

Send them as one page_options object on any endpoint that loads the page — every one but /lighthouse. They are included in the operation's usual credits.

POST /markdown

{
  "url": "https://example.com/blog/launch",
  "page_options": {
    "block_ads": true,
    "block_cookie_banners": true,
    "remove_selectors": [".newsletter-signup", "#related"]
  }
}

Options

Name
wait_for_selector
Type
string
Description
Wait for an element matching this CSS selector before reading the page, up to 10 seconds — for a single-page app whose content arrives after the page has loaded. An element that never appears fails the request, and costs nothing.
Name
delay
Type
integer
Description
Milliseconds to wait after the page has settled, 0–10000. For content, or console errors, that arrive on a timer.
Name
block_ads
Type
boolean
Description
Block the major ad networks' requests before the page loads, and remove the slots they would have filled. Pages with ads usually load much faster with it.
Name
block_cookie_banners
Type
boolean
Description
Remove the cookie-consent banners of the platforms that serve most of the web's — OneTrust, Cookiebot, Usercentrics, Didomi, Quantcast, Sourcepoint and more — including the cookie policy text their preference panels put in the page. Nothing is clicked, so no consent is given on your behalf.
Name
remove_selectors
Type
array
Description
CSS selectors of your own elements to remove — a chat widget, a promo bar, a related-posts list. Up to 50.

Removed, not hidden

Everything these options take out is removed from the page, not hidden in it, so it is absent from the HTML and from everything derived from it. On onetrust.com, block_cookie_banners takes about 16,000 characters of cookie policy out of the page's text; on elmundo.es, block_ads roughly halves the time the page takes to load.

To change only how a screenshot looks — hide an element in the image but keep it in the HTML — use screenshot_options.hide_selectors instead.

Stored results

Page options are part of what identifies a result: a page with its ads removed is a different result from the page with them, and each is stored and reused on its own. An option set to its default is the same request as one that leaves it out.

Why not /lighthouse

A Lighthouse audit loads the page for itself, and measures that load. Waiting on it, blocking its requests or trimming it would change the very scores the audit reports, so /lighthouse refuses page_options with invalid_options. In a /scrape, they apply to every operation except the audit.