Glossary
The web data glossary
Short, exact definitions of the terms you meet when you fetch, render, extract from and measure web pages. Every entry ends with a free tool that shows the term on a real page.
Web pages as model input
- Keyword extractionPulling the terms and phrases that say what a page is about out of its text — from any URL.Read
- Markdown for LLMsConverting a web page to Markdown before a model reads it — fewer tokens, same structure.Read
- Context windowHow much text a language model can take in at once — the budget every fetched page spends from.Read
- ChunkingSplitting documents into pieces small enough to embed and retrieve — best done along the headings.Read
- EmbeddingsLists of numbers that place text by meaning, so similar passages can be found without matching words.Read
- llms.txtA Markdown file at a site's root that tells language models what the site is and where its best content lives.Read
Metadata and page capture
- Open GraphThe og: meta tags that decide the title, description and image a link preview shows.Read
- og:imageThe one tag that decides the picture on every shared link to your page.Read
- Twitter CardsThe twitter: meta tags that shape how a link looks on X — and how they fall back to Open Graph.Read
- JSON-LDStructured data in a script tag — the schema.org facts search engines read for rich results.Read
- Full-page screenshotAn image of the whole page, top to bottom — not just the part that fits in the window.Read
- Device pixel ratioHow many physical pixels a screen uses for one CSS pixel — why retina screenshots are twice the size.Read
- Cookie consent bannerThe consent pop-up European privacy law puts on most sites — and in most screenshots and scraped text.Read
Page performance and errors
- Core Web VitalsGoogle's three user-experience metrics — loading, responsiveness and visual stability — and their thresholds.Read
- Largest Contentful Paint (LCP)When the biggest image or text block in the viewport finishes rendering — the loading Core Web Vital.Read
- Cumulative Layout Shift (CLS)How much the page's content jumps around unexpectedly while you read it — the visual-stability Core Web Vital.Read
- Interaction to Next Paint (INP)How long the page takes to visibly respond to a click, tap or key press — the responsiveness Core Web Vital.Read
- Total Blocking Time (TBT)How long long JavaScript tasks block the main thread during load — Lighthouse's heaviest-weighted metric.Read
- Speed IndexHow quickly the visible part of the page fills in during load, averaged over time.Read
- Time to First Byte (TTFB)How long before the server's response starts arriving — the floor under every other loading metric.Read
- Unhandled promise rejectionA JavaScript promise that failed with nothing there to catch it — the error that breaks features silently.Read
Fetching and rendering
- Rendered DOMThe page as it exists after the browser has run its JavaScript — what a visitor sees, and what curl never returns.Read
- Client-side renderingWhen the browser, not the server, builds the page from JavaScript — which is why curl gets an empty shell.Read
- HydrationAttaching JavaScript to HTML the server already rendered, so a static page becomes an interactive app.Read
- Headless browserA real browser engine run without a window, driven by code — how you read, screenshot and measure pages at scale.Read
- robots.txtThe file at a site's root that tells automated clients which paths they may fetch — a request, not a lock.Read
- networkidleThe wait condition that treats a page as loaded once its network traffic goes quiet.Read
- DOMContentLoadedThe event fired once the HTML is parsed and deferred scripts have run — before images, and often before content.Read
- Residential proxySending a request out through a home internet connection's IP address instead of a datacentre's.Read
- Bot detectionHow sites decide whether a visitor is a person or a program — and what they do about it.Read
- Webhook signatureA keyed hash sent with each webhook so the receiver can prove who sent it and that nothing changed.Read
- Idempotency keyA unique value sent with a request so retrying it can never do the work — or the charge — twice.Read
- Cache TTLHow long a stored result stays fresh enough to reuse — the knob between cost and freshness.Read
- SSRFTricking a server into fetching a URL on its own internal network — the first risk of any 'fetch this URL' feature.Read
Turn any URL into clean data in minutes.
Free plan, no card. Confirm your email and your API key is live — you'll be making real requests in minutes.