Glossary
Keyword extraction
What a page is about, in ten phrases — and what a keyword list can and can't tell you.
By Roger Campos · Last updated: September 2026
TL;DR
Keyword extraction is the automatic selection of the words and phrases that best describe what a text is about. Applied to a web page, it reads the page's main content — not its navigation or footer — and returns a short ranked list of topics, used for tagging, clustering, search and SEO research.
Free plan, no credit card. 1,000 credits a month.
How it works
The methods
Every method has the same two steps: pick candidate words and phrases from the text, then rank them. They differ in how they rank.
| Method | Ranks by | Good at | Weak at |
|---|---|---|---|
| Word frequency | How often a term appears | Speed; no setup | Returns "the", "page" and the site name without heavy filtering |
| TF-IDF | Frequency here vs. across a corpus | Terms distinctive to one page among many | Needs a corpus; single words over phrases |
| RAKE, YAKE | Word co-occurrence and position statistics | Multi-word phrases, no training | Near-duplicate phrases; no sense of topic |
| KeyBERT-style | Embedding similarity to the whole text | Phrases that match the page's meaning | Needs an embedding model; can favour generic phrases |
| Language model | Asked to judge what the page is about | Topic-level phrases, deduplicated, any language | Costs a model call; not reproducible byte for byte |
In practice
Getting keywords from a URL
On a web page the method matters less than the input. Run any of them on the raw HTML and the top keywords are the menu, the cookie notice and the footer — the text that is on every page of the site. The first step is always to reduce the page to its main content, after JavaScript has rendered it.
Then be clear about what you get. Keywords extracted from a page say what the page is about. They are not what it ranks for, and they carry no search volume, difficulty or traffic — those come from search data, which a page's text doesn't contain.
URLpipe
Keyword extraction with URLpipe
/keywords renders the page in real Chrome, reduces it to its readable text — navigation, sidebars and page chrome dropped — and asks a language model for the 5–15 keywords and phrases that describe it, most relevant first. Near-duplicates are merged ("documentation system" and "documentation principles" become one), the site name and cookie notices are ignored, and the keywords come back in the page's own language.
curl -X POST https://urlpipe.dev/keywords \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://en.wikipedia.org/wiki/Search_engine_optimization", "sync": true}'import requests
urls = ["https://example.com/", "https://example.com/pricing", "https://example.com/blog/launch"]
for url in urls:
keywords = requests.post(
"https://urlpipe.dev/keywords",
headers={"Authorization": "Bearer YOUR_API_KEY"},
json={"url": url, "sync": True},
).json()
print(url, keywords[:5])The response is a JSON array of strings. A call costs 15 credits because it calls a model; a repeat within max_age is free. For bulk work where you want your own ranking — TF-IDF across a whole site, say — fetch each page's text from /markdown at 1 credit and rank it yourself. The free keyword extractor runs the same endpoint on any URL, no signup.
Try it
Keyword extraction, on a page you choose
Related terms
FAQ
Frequently asked questions
How do I get the keywords of a website from its URL?
Does keyword extraction show search volume?
Can I see a competitor's keywords this way?
See it on your own pages.
Free plan, no card. Confirm your email and your API key is live — you'll be making real requests in minutes.