cURL · Code recipe
Get a page's metadata and Open Graph tags in cURL
Read the title, description and share image of any page, in any shell with curl with curl. Every program on this page runs as it stands — each one was run against a stub of the API before it was published.
By Roger Campos · Last updated: September 2026
TL;DR
To read a page's title, description, Open Graph image and other metadata in cURL, POST its URL to https://urlpipe.dev/meta with sync set to true and parse the JSON with jq. It answers with nine fields, any of which can be null, for 5 credits: it is one of the three endpoints that call a language model.
Free plan, no credit card. 1,000 credits a month.
One POST to /meta renders the page and returns what it says about itself: title, description, language, main image, favicon, author, feed, first publication date and extra author details. URLs come back absolute, resolved against the page.
A language model reads the page's metadata declarations — Open Graph, Twitter cards, JSON-LD, plain tags — and settles conflicts by a fixed order: og:title, then twitter:title, then <title>, then the <h1>. A field the page never declares is null, never a guess. That is why it costs 5 credits where a page fetch costs 1 credit. In cURL, jq gives you the fields; the Open Graph guide covers which tags each platform reads.
Setup
Before you start
curl is on every Mac and almost every Linux box. jq reads the JSON this endpoint answers with; install it from your package manager if jq --version finds nothing.
curl --version | head -1
jq --version # brew install jq / apt install jq
export URLPIPE_API_KEY="your_api_key"
The request
Print the title, description and main image
"sync": true keeps the request open until the result is ready. -w '%{http_code}' hands you the status and -o parks the body in a file, so an error message is printed, not decoded or saved as if it were the result. --max-time stops a page that never finishes loading from hanging the script. // is jq's fallback for a field that came back null.
#!/usr/bin/env bash
set -euo pipefail
response=$(mktemp)
trap 'rm -f "$response"' EXIT
# -w prints the status; -o keeps the body out of the way until the status is known.
status=$(curl -sS --max-time 90 -o "$response" -w '%{http_code}' \
https://urlpipe.dev/meta \
-H "Authorization: Bearer $URLPIPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "sync": true}')
if [ "$status" != 200 ]; then
echo "URLpipe answered $status: $(cat "$response")" >&2
exit 1
fi
jq -r '"Title: \(.title // "none")",
"Description: \(.description // "none")",
"Image: \(.main_image_url // "none")"' "$response"
Run it: bash page_metadata.sh
Given the example response on the docs page, it prints:
Title: Example Domain
Description: Illustrative examples in documents.
Image: https://example.com/cover.jpgAsync
The async variant: a token, a webhook and a poll
Leave out sync and the answer is a token, straight away; jq -r .token pulls it out without quotes. The loop polls every two seconds and the case covers the four answers GET /result/:token can give.
#!/usr/bin/env bash
set -euo pipefail
response=$(mktemp)
trap 'rm -f "$response"' EXIT
API=https://urlpipe.dev
AUTH="Authorization: Bearer $URLPIPE_API_KEY"
# No "sync": the request is accepted at once and the work carries on without you.
status=$(curl -sS -o "$response" -w '%{http_code}' "$API/meta" \
-H "$AUTH" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example.com",
"report_to": "https://your-app.com/webhooks/urlpipe",
"labels": {"customer": "acme"}
}')
if [ "$status" != 200 ]; then
echo "URLpipe answered $status: $(cat "$response")" >&2
exit 1
fi
token=$(jq -r .token "$response")
echo "Accepted $token"
# The result is POSTed to report_to when it is ready. Polling by token is the
# other way to collect it: no endpoint needed, and a backup for the webhook.
for _ in $(seq 60); do
status=$(curl -sS -o "$response" -w '%{http_code}' "$API/result/$token" -H "$AUTH")
[ "$status" = 202 ] || break # 202 means still processing
sleep 2
done
case "$status" in
200) ;;
202) echo "Still processing after two minutes; try the token again later." >&2; exit 1 ;;
422) echo "The analysis failed: $(jq -r .error "$response")" >&2; exit 1 ;;
410) echo "The result is past the 30-day window; send the request again." >&2; exit 1 ;;
*) echo "URLpipe answered $status: $(cat "$response")" >&2; exit 1 ;;
esac
jq -r '"Title: \(.title // "none")",
"Description: \(.description // "none")",
"Image: \(.main_image_url // "none")"' "$response"
Run it: bash page_metadata_async.sh
Errors
Handle errors and retries
-D writes the response headers to a file, which is where Retry-After is read from. A 401 answers in plain text, so it is decided on the status alone; jq reads the error code of everything else.
#!/usr/bin/env bash
set -euo pipefail
response=$(mktemp)
headers=$(mktemp)
trap 'rm -f "$response" "$headers"' EXIT
# urlpipe PATH JSON: POST a sync request, leaving the result in $response.
# Retries the two 429s that clear by themselves; fails with a message otherwise.
urlpipe() {
local attempt status code detail retry_after
for attempt in 0 1 2 3 4; do
status=$(curl -sS --max-time 90 -o "$response" -D "$headers" -w '%{http_code}' \
"https://urlpipe.dev$1" \
-H "Authorization: Bearer $URLPIPE_API_KEY" \
-H "Content-Type: application/json" \
-d "$2")
case "$status" in
200) return 0 ;;
401) echo "URLpipe: 401: the API key is missing or wrong. Check URLPIPE_API_KEY." >&2; return 1 ;;
esac
# A body that is not JSON leaves both empty.
code=$(jq -r '.error // ""' "$response" 2>/dev/null || true)
detail=$(jq -r 'if .message then "\(.error): \(.message)" else .error end' "$response" 2>/dev/null || true)
if [ "$status" = 429 ] && [ "$code" = rate_limited ]; then
# Sending too fast: Retry-After says how long the window has left.
retry_after=$(awk 'tolower($1) == "retry-after:" { print $2 + 0 }' "$headers")
sleep "${retry_after:-1}"
elif [ "$status" = 429 ] && [ "$code" = concurrency_limit ]; then
# Every parallel slot on your plan is busy with your own requests.
sleep $((2 ** attempt))
elif [ "$status" = 504 ]; then
# Still running on our side; the token collects it from GET /result/:token.
echo "URLpipe: 504 processing_timeout: collect it later with token $(jq -r .token "$response")" >&2
return 1
else
# 403 email_unverified, 422 (a bad parameter, or a page that would not load),
# 429 quota_exceeded: sending the same request again gets the same answer.
echo "URLpipe: $status: $detail" >&2
return 1
fi
done
echo "URLpipe: 429: still refused after 5 attempts" >&2
return 1
}
urlpipe /meta '{"url": "https://example.com", "sync": true}' || exit 1
jq -r '"Title: \(.title // "none")",
"Description: \(.description // "none")",
"Image: \(.main_image_url // "none")"' "$response"
Run it: bash page_metadata_errors.sh
Details
What to know about /meta
- Any field can be
nullwhen the page does not have it — code for that, as the program does. - There is no
canonicalfield; the nine fields are the whole response. - Image and favicon URLs that are data URIs come back as
nullrather than as a blob. - Pages over 10 MB of HTML are refused before the model sees them.
FAQ
Frequently asked questions
Which fields does /meta return?
Why does metadata cost more than fetching the HTML?
Is AI processing done in the EU?
Do I need an SDK to call URLpipe from cURL?
Get a key and run it.
Free plan, no card. Paste your key into URLPIPE_API_KEY and every program on this page runs as it is.