The keywords of any URL, ranked

The request

One POST, one bearer token

curl -X POST https://urlpipe.dev/keywords \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/blog/branch-prediction", "sync": true}'

import requests

res = requests.post(
    "https://urlpipe.dev/keywords",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={"url": "https://example.com/blog/branch-prediction", "sync": True},
)

const res = await fetch("https://urlpipe.dev/keywords", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ url: "https://example.com/blog/branch-prediction", sync: true }),
})

$ch = curl_init("https://urlpipe.dev/keywords");
curl_setopt_array($ch, [
    CURLOPT_POST => true,
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_HTTPHEADER => [
        "Authorization: Bearer YOUR_API_KEY",
        "Content-Type: application/json",
    ],
    CURLOPT_POSTFIELDS => json_encode(["url" => "https://example.com/blog/branch-prediction", "sync" => true]),
]);

$response = curl_exec($ch);

[
  "branch prediction",
  "pipeline stalls",
  "branch target buffer",
  "speculative execution",
  "loop unrolling"
]

The keywords

What comes back

• The subject first. The first keyword is what the whole page is about, in the most specific term that still covers all of it. The rest follow by relevance.
• As many as the page has topics. Between 5 and 15: a short page, or a page about one thing, has 5 to 8. The list is not padded to 15.
• Terms, not sentences. One to three words each, taken from the page's own text, each about something different from the others.
• In the page's language. A Spanish page gets Spanish keywords.
• Without the noise. No author names, cookie notices, prices, dates or identifiers, and not the site's own name unless the page is about the site.

Use cases

What people build with it

• Competitor and SEO research. The topics a competitor's pages are written around, page by page, to seed a brief or find gaps.
• Auto-tagging. Tags for articles, bookmarks and knowledge-base entries as they are added.
• Clustering URLs. Group a list of pages by what each is about, with labels to keep each batch apart.
• Related content and search. Keywords as index terms for internal search or "more like this" links.

const res = await fetch("https://urlpipe.dev/keywords", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ url: "https://example.com/blog/post", sync: true }),
})

const tags = await res.json() // ["branch prediction", "pipeline stalls", …]

Limits

What it does not do

• No search volumes or rankings. It tells you what a page is about, not how often anyone searches for it.
• No scores. The keywords come back in order of relevance, without a number on each.
• One page per request. For a whole site, send each page you care about.
• Up to 10 MB of HTML, and public pages only.