Buyer's guide

The best HTML-to-Markdown APIs for LLMs & RAG

Feeding web pages to a language model works best when the HTML is turned into clean Markdown first. These seven APIs and tools do it well in 2026 — here's what each is best for, how they're priced, and the honest trade-offs.

By Roger Campos · Last updated: July 2026

TL;DR

For per-URL extraction that needs more than Markdown, URLpipe is the most versatile single API. For crawling entire sites, use Firecrawl. For the fastest one-off Markdown call, use Jina Reader. ScrapingBee handles anti-bot scraping, Diffbot does knowledge-graph extraction, Apify orchestrates reusable scrapers, and Crawl4AI is the open-source, self-hosted option.

Start free See the API

Free plan, no credit card. 50 AI + 500 web operations / month.

What to look for

How to choose an HTML-to-Markdown API

Before comparing tools, weigh these six criteria against your own use case. The right pick usually comes down to scope (one URL or a whole site) and how much data you need beyond text.

Markdown quality

Does it strip nav, ads and boilerplate and return clean, well-structured Markdown that chunks cleanly for embeddings?

JavaScript rendering

Modern sites are client-rendered. The tool must run a real browser, or it'll return empty shells for single-page apps.

Scope

Do you need one URL at a time, or a full-site crawl? Per-URL APIs are simpler; crawlers are built for breadth.

Data beyond Markdown

Sometimes you also need metadata, a summary, a screenshot or a performance audit. Fewer vendors is fewer integrations to maintain.

Pricing model

Credits per page, tokens per character, or flat operations? Match the meter to your traffic — and check how caching is billed.

Hosting & licensing

Managed vs self-hosted, data region, and open-source licensing can all be hard requirements.

At a glance

The 7 best HTML-to-Markdown APIs compared

Tool	Renders JS	Full-site crawl	Free tier	Standout
URLpipe	Yes	No	50 AI + 500 web ops/mo	8 data types per URL
Firecrawl	Yes	Yes	1,000 credits/mo	Crawl + schema extraction
Jina Reader	Yes	Partial	20 RPM + ~10M tokens	Zero-signup prefix
ScrapingBee	Yes	Partial	1,000 credits	Proxies + anti-bot
Diffbot	Yes	Yes	~10,000 credits (trial)	Knowledge graph
Apify	Yes	Yes	~$5 credits/mo	Actor marketplace
Crawl4AI	Yes	Yes	Free (self-host)	Self-hosted, open source

“Full-site crawl” marked partial means single-URL first with limited multi-page support. Pricing and limits change — confirm current numbers on each vendor's site.

The tools

Each option in depth

Listed with the most versatile per-URL API first. Every entry includes what it's best for, pricing, strengths and an honest watch-out.

URLpipe

That's us

One API that turns a single URL into eight kinds of clean data.

Best for: Per-URL extraction that needs more than Markdown.
Pricing: Free: 50 AI + 500 web operations/mo, no card. Then Unlimited (custom).

Strengths

Markdown plus metadata, AI summaries, keywords, Lighthouse audits, screenshots and JS console logs — one bearer token
Renders JavaScript with headless Chrome
Cache hits are free and never count toward quota (default 7-day freshness)
Sync responses or async webhook delivery
EU-hosted, per-project API keys and request history

Watch out

Single-URL only — it doesn't crawl entire sites or do schema-based extraction, and there's no self-serve mid-tier between Free and Unlimited.

Firecrawl

The go-to crawl-and-extract platform for RAG at scale.

Best for: Crawling whole sites and structured extraction across many pages.
Pricing: Free: 1,000 credits/mo. Paid $16–$599/mo (~5k–1M pages).

Strengths

Full-site crawl, URL mapping and schema-based structured extraction
Clean Markdown output tuned for LLM pipelines
Large self-serve pricing ladder up to millions of pages
Official Python/Node SDKs and LangChain / LlamaIndex integrations
Open-source, self-hostable core

Watch out

Credits are billed per page and don't roll over; no Lighthouse audits, console capture, summaries or keyword endpoints.

Jina Reader

Prepend r.jina.ai/ to any URL and get Markdown back.

Best for: The lowest-friction URL-to-Markdown call and PDF reading.
Pricing: Free: 20 RPM no key; ~10M free tokens with a key. Then token-based.

Strengths

Zero-signup prefix — try any URL instantly in a browser
Very large free token allowance for new accounts
Native PDF parsing and automatic image captioning (vision models)
CSS-selector targeting, viewport and proxy controls
Part of Jina's wider AI-search stack (embeddings, reranker, search)

Watch out

Token billing scales with page length; no screenshots, Lighthouse or console capture; cache holds results only ~5 minutes.

ScrapingBee

A scraping API with strong proxy and anti-bot handling.

Best for: Hard-to-scrape sites that need rotating proxies and stealth.
Pricing: Free: 1,000 credits. Paid $49–$249/mo (250k–3M credits).

Strengths

Rotating premium proxies and JavaScript rendering
Converts pages to clean Markdown or plain text, LLM-ready
Good for sites with aggressive bot protection
Mature, well-documented API

Watch out

Credit costs multiply with JS rendering (5×) and premium domains (10–75×); focused on scraping, not multi-data-type extraction.

Diffbot

AI extraction with a knowledge graph behind it.

Best for: Structured entity extraction and knowledge-graph enrichment.
Pricing: Free trial: ~10,000 credits. Paid from $299/mo.

Strengths

Automatic classification of pages into articles, products, etc.
Structured, typed output and entity extraction
Backed by a large web knowledge graph
Strong for data-enrichment use cases

Watch out

Pricing starts high for small projects; more an extraction platform than a lightweight Markdown API.

Apify

A marketplace and platform for scrapers (Actors).

Best for: Orchestrating reusable scrapers with scheduling and storage.
Pricing: Free: ~$5 platform credits/mo. Paid from $29/mo.

Strengths

Large marketplace of ready-made scraping Actors
Built-in scheduling, storage, retries and monitoring
Competitive at high volume when infrastructure is included
Flexible — build custom Actors or reuse community ones

Watch out

More of a platform to assemble than a single Markdown endpoint; a learning curve for one-off jobs.

Crawl4AI

Open-source, self-hostable crawler with LLM-ready Markdown.

Best for: Teams that want to self-host with no per-page cost.
Pricing: Free — open source (Apache-2.0). You pay only for infrastructure.

Strengths

Apache-2.0 license — commercial-friendly, no per-page fees
Clean Markdown output designed for LLMs
Runs on Docker with Playwright; integrates via LiteLLM
Full control over the crawling stack

Watch out

You run and maintain it yourself — no managed service, scaling or support out of the box.

Recommendations

Which one should you pick?

Feed single pages to an LLM and also need summaries, audits or screenshots

URLpipe

Crawl and structure an entire website for a RAG index

Firecrawl (or Diffbot / Apify)

Quickest possible one-off URL-to-Markdown, or reading PDFs

Jina Reader

Scrape sites with heavy bot protection or geo-restrictions

ScrapingBee

Extract typed entities and enrich against a knowledge graph

Diffbot

Reuse ready-made scrapers with scheduling and storage

Apify

Self-host with no per-page cost and full control

Crawl4AI

FAQ

Frequently asked questions

What is the best HTML-to-Markdown API?

There's no single winner — it depends on scope. For turning individual URLs into clean Markdown plus other data (metadata, summaries, keywords, Lighthouse, screenshots, console logs), URLpipe is the most versatile single API. For crawling entire sites and schema-based extraction at scale, Firecrawl leads. For the lowest-friction one-off Markdown call, Jina Reader is hard to beat. For heavy anti-bot scraping, ScrapingBee; for self-hosting, the open-source Crawl4AI.

Why convert HTML to Markdown for LLMs?

Raw HTML is full of navigation, scripts, styling and boilerplate that waste tokens and confuse models. Markdown keeps the meaningful content and its structure — headings, lists, links — in a compact form that's ideal for prompting and for chunking into a vector store. Clean Markdown improves both retrieval quality and cost in RAG pipelines.

What's the difference between an HTML-to-Markdown API and a web scraper?

A general web scraper returns whatever you target — often raw HTML or custom-selected fields — and leaves cleanup to you. An HTML-to-Markdown API specializes in producing clean, readable Markdown of a page's main content, boilerplate removed, ready for an LLM. Several tools do both; the distinction is whether clean Markdown is a first-class output or an afterthought.

Which HTML-to-Markdown API has the best free tier?

It depends how you measure. Jina Reader offers a very large free token allowance and a no-signup prefix. URLpipe's free plan (50 AI + 500 web operations/month, no credit card) is built for real low-volume production and makes cache hits free. Firecrawl gives 1,000 credits/month. Crawl4AI is free to self-host. Match the free tier's unit to your traffic pattern.

Do these APIs handle JavaScript-rendered sites?

The managed options here (URLpipe, Firecrawl, Jina Reader, ScrapingBee, Diffbot, Apify) render JavaScript with a real headless browser, so single-page apps and client-rendered content work. Crawl4AI does too, via Playwright, when you self-host it.

Is there a good open-source HTML-to-Markdown option?

Yes. Crawl4AI is an Apache-2.0 crawler with LLM-ready Markdown output that you can self-host on Docker with Playwright — no per-page fees, full control. The trade-off is that you operate and scale it yourself, without a managed service or support.

Keep comparing

URLpipe vs Firecrawl URLpipe vs Jina Reader

Try the most versatile one on the list.

URLpipe turns any URL into Markdown, metadata, summaries, keywords, Lighthouse audits, screenshots and console logs — one API, one token. Free plan, no card.

Start free Email the team