Skip to main content
Open navigation URLpipeDocs
Start free

Extract

Describe the object you want — each field with a name, a type and what to take from the page — and get it back filled from the page: product details, prices, contact data, job listings, event dates, lists of items with their own attributes. Any value the page does not give comes back null; nothing is guessed.

This is an AI operation: the whole page — text, links, tables and its structured data — is read by a language model with reasoning, held to your schema. It costs 24 credits per call. See Credits. When the standard metadata is all you need, /meta gives it for one credit and no model.

POST/extract

Extract data

Body parameters

  • Name
    url
    Type
    string
    Required
    Required
    Description
    The absolute URL of the page to process. Rendered with headless Chrome, so JavaScript runs and redirects are followed. It must not include a username or password (https://user:pass@example.com).
  • Name
    fields
    Type
    array
    Required
    Required
    Description
    The fields to fill, in the order you want them back. Each has a name, a type and, optionally but worth writing, a description — see Fields.
  • Name
    instructions
    Type
    string
    Description
    Guidance that applies to every field, up to 1,000 characters — Prices in EUR, including VAT or Only the first product on the page.
  • Name
    page_options
    Type
    object
    Description
    Wait for the page, and remove ads, cookie banners or your own elements from it before it is read — gone from this result, not merely hidden. See Page options.
  • Name
    residential
    Type
    boolean
    Description
    Fetch the page from a residential exit — an address on a home broadband line rather than one in a datacentre. Reach for it when a site serves you less than it serves a browser, or nothing at all. Defaults to false. Adds 25 credits per page fetch on top of what the operation costs, and its results are kept separate from the ordinary ones.
  • Name
    report_to
    Type
    string
    Description
    Webhook URL — an http or https address URLpipe POSTs the result to when it's ready. Optional: without it we deliver to your project's default endpoint if it has one, and otherwise send no webhook at all — the result still waits for you at GET /result/:token. A value we cannot deliver to returns 422. Ignored on a sync=true request. Deliveries can be signed so your endpoint can verify they came from us.
  • Name
    sync
    Type
    boolean
    Description
    Process the request synchronously, returning the result inline in the response. Defaults to false (async: return a token now, and either receive the result at a webhook or fetch it with GET /result/:token). See Async & sync modes for the full contract.
  • Name
    max_age
    Type
    string | integer
    Description
    How fresh a cached result must be to be accepted. Either an integer number of seconds (3600) or a duration string of the form "<number> <unit>" — units s/min/h/d/w (e.g. "2 hours", "3 days", "30m"). Defaults to 7 days, clamped to a max of 30 days; 0 always bypasses the cache. See Caching for all accepted units.
  • Name
    labels
    Type
    object
    Description
    Your own keys to find and account for this request by — a client, a project, a campaign: {"client": "acme"}. Returned with the result, in the webhook and in the X-Labels header, and your dashboard filters history and totals credits by them. Up to 16 keys; string values. See Labels.

Response

Content type application/json — an object with exactly your fields as keys, in your order. A value the page does not give, or that does not fit its type, is null; a list with nothing in it is [].

POST/extract
curl -X POST https://urlpipe.dev/extract \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "fields": [{"name":"name","type":"string","description":"The product's name"},{"name":"price","type":"number","description":"The current price, without the currency"},{"name":"currency","type":"string","description":"ISO 4217 code of the price's currency"},{"name":"in_stock","type":"boolean","description":"Whether it can be bought now"},{"name":"images","type":"array","items":"url","description":"Product photos"}]}'
Response
{
  "name": "Trail Runner 3",
  "price": 129.95,
  "currency": "EUR",
  "in_stock": true,
  "images": [
    "https://example.com/img/trail-runner-3-side.jpg",
    "https://example.com/img/trail-runner-3-top.jpg"
  ]
}

Fields

Each field is an object. name is the key it comes back under: letters, digits and underscores, starting with a letter or underscore. The description is what the model reads to decide which value on the page you mean, so say it the way you would to a person: The sale price if there is one, otherwise the regular price.

  • Name
    string
    Type
    type
    Description
    Text, as the page writes it.
  • Name
    number
    Type
    type
    Description
    A number, without currency symbols, units or separators.
  • Name
    integer
    Type
    type
    Description
    A whole number.
  • Name
    boolean
    Type
    type
    Description
    true or false.
  • Name
    date
    Type
    type
    Description
    An ISO 8601 date, 2026-10-08.
  • Name
    datetime
    Type
    type
    Description
    An ISO 8601 date and time, with its offset when the page gives one.
  • Name
    url
    Type
    type
    Description
    An absolute URL. Relative links on the page are resolved against it.
  • Name
    email
    Type
    type
    Description
    An email address.
  • Name
    object
    Type
    type
    Description
    A group of values, with its own fields.
  • Name
    array
    Type
    type
    Description
    A list of every match on the page, in page order. Its items say what it holds: a type on its own ("items": "url") or an object with fields, for a list of things with several attributes each.
A list of objects
{
  "url": "https://example.com/careers",
  "fields": [
    { "name": "company", "type": "string" },
    { "name": "jobs", "type": "array", "description": "Every open position", "items": {
        "type": "object",
        "fields": [
          { "name": "title", "type": "string" },
          { "name": "location", "type": "string", "description": "City, or Remote" },
          { "name": "posted_on", "type": "date" },
          { "name": "apply_url", "type": "url" }
        ]
    } }
  ]
}

Up to 50 fields in all, nested at most 3 levels deep. A schema that breaks these rules is refused with invalid_options and a message naming the field, before anything is fetched or spent.

How it works

  • The page is rendered in a real browser, so content added by JavaScript is there.
  • A URL that serves a PDF or a text file is read too, from its text — see Documents & files.
  • The whole page is read — not only its article text — reduced to its text, links, tables, image alt text and its structured data (JSON-LD, microdata, meta tags), where prices, dates and identifiers often are.
  • A model with a short reasoning step fills your schema. It is told to leave a value null rather than guess, and its answer is held to your schema: every key there, nothing else, and a value that does not fit its type turned into null.
Results are stored and reused like every other: the same URL with the same fields and instructions within max_age is answered for free. Change a field, its order or its description, and it is a new extraction.

Responses

Whatever the status, the response carries metadata headers: the result token, whether it was served from cache and how old that result is, how long we took, what it cost in credits, and the allowance you have left.

Status
When
Body
200 OK
The request succeeded.
Sync: the result, in this endpoint's format (see Response above). Async: a job token and the request's labels — { "token": "…", "status": "accepted", "labels": {} }.
422 Unprocessable Entity
The analysis failed, or a parameter was invalid (a bad max_age, a screenshot_options value out of range, labels that break the rules, or a report_to we will not deliver to).
{ "error": "<message>" }
429 Too Many Requests
Three causes, told apart by the error field: concurrency_limit (too many of your requests already running), rate_limited (sending too fast), or quota_exceeded (Free plan only — out of credits with no card on file to bill the extra to, and checked only on a cache miss; a paid plan keeps serving at the overage rate and is never refused for credits).
All three carry "error" and "message". Extra fields: concurrency_limit → "limit", "running" · rate_limited → "retry_after" · quota_exceeded → "limit", "used", "needed", "resets_at".
504 Gateway Timeout
Sync only: the analysis didn't finish within 60s. It keeps running — fetch it via GET /result/:token.
{ "error": "processing_timeout", "token": "…" }
401 Unauthorized
Missing or invalid API key.
{ "error": "invalid_api_key", "message": "…" }

Try it live — no API key needed

Run this endpoint against any URL right in your browser.

Open tool