Guide
What llms.txt is and how to write one
llms.txt is a Markdown file at the root of a site that tells a language model what the site is and where the useful pages are. Here's the format, an example, and an honest look at who reads it.
By Roger Campos · Last updated: September 2026
TL;DR
llms.txt is a proposed standard: a Markdown file at /llms.txt with the site's name as an H1, a one-paragraph summary in a blockquote, and H2 sections listing links to the pages a model should read. It helps agents and coding tools that fetch it on purpose. Google has said its Search does not use it, so don't expect it to change rankings or AI Overviews.
Free plan, no credit card. 1,000 credits a month.
The idea
A table of contents for language models
llms.txt is a Markdown file at the root of a website — /llms.txt — that tells a language model, in a few hundred words, what the site is and which pages are worth reading.
The problem it addresses is real. A model or an agent that wants to understand a site has two bad options: read the HTML, which is mostly navigation and markup, or guess which of hundreds of pages matter. A context window is small; a site is large. A short, curated file that says "here's what we are, and here are the ten pages that explain it" is a better starting point than either.
It was proposed by Jeremy Howard in September 2024. It is a proposal — a convention some sites and tools have adopted — not a standard any body has ratified, and not something search engines have agreed to read.
The format
What goes in an llms.txt file, section by section
The file is plain Markdown in a fixed order, so it's readable by people and parseable by programs. Only the first element is required:
- 1An H1 with the site or project name. The one required line.
- 2A blockquote summary. One short paragraph with the key facts a reader needs to understand everything below — what it is, who it's for, what makes it different.
- 3Optional detail in ordinary paragraphs or lists: how to use it, important caveats, conventions. No headings here.
- 4H2 sections of links. Each is a list of
[name](url): notesentries pointing at the pages worth reading, ideally in a Markdown version. - 5An "Optional" section, by convention, for secondary links an agent can skip when it needs a shorter context.
# Example Co
> Example Co is an invoicing API for freelancers. It creates, sends and reconciles
> invoices in 30 currencies. REST, JSON, bearer-token auth.
Every endpoint is idempotent with an Idempotency-Key header. Amounts are integers
in the currency's minor unit.
## Docs
- [Quickstart](https://example.com/docs/quickstart.md): first invoice in five minutes
- [Invoices API](https://example.com/docs/invoices.md): create, send, void
- [Webhooks](https://example.com/docs/webhooks.md): events and signature verification
## Optional
- [Changelog](https://example.com/changelog.md)The proposal also suggests serving a Markdown version of each page at the same URL with .md appended, so the links can point at clean text. Many sites also publish an llms-full.txt — the full text of the linked pages in one file — which isn't part of the proposal but is a common companion.
A real one
An example you can read
Ours is at urlpipe.dev/llms.txt. It follows the shape above: the name, a one-paragraph summary of what the API does, then sections for the endpoints, features, pricing, the free tools, the guides, key pages and comparisons. Two choices in it are worth copying:
- It's generated, not hand-written. The endpoint list, prices and page links come from the same data the website is built from. A hand-kept llms.txt drifts from the product within weeks — and a model will repeat a stale price with total confidence.
- It says what the product doesn't do. A model asked "can this do X?" answers from what it read. An explicit boundary gets you an accurate "no" instead of a hopeful "probably".
Honestly
Who actually reads llms.txt?
Less than the enthusiasm around it suggests. Google's guide to its generative AI features is explicit: Google Search doesn't use llms.txt or similar AI text files, and creating one neither helps nor harms a site in Search, AI Overviews included.
Where it does help is with tools a person points at a site on purpose. A developer who tells a coding assistant "read the docs at example.com", an agent given a domain to research, a docs tool that ingests a site — these fetch /llms.txt when it exists, because it's the cheapest way to find the right pages. That's a narrower audience than search, and a valuable one if developers or agents use your product.
| File | Tells automated clients | Enforced? |
|---|---|---|
| robots.txt | What they may not fetch | Voluntary, widely honoured |
| sitemap.xml | Which URLs exist and when they changed | A hint for search engines |
| llms.txt | What's worth reading, and in what order | Read only by tools that look for it |
The pages it links to
Serving Markdown versions of your pages
An llms.txt full of links to HTML pages still makes every tool do the hard part — rendering, stripping the navigation, converting. The proposal's suggestion is to serve each page as Markdown at a predictable URL: /docs/quickstart.md beside /docs/quickstart. There are three common ways to produce them:
- From the source. If your docs are written in Markdown or MDX, publish the source alongside the built page. The cleanest option, and the one static-site generators make easy.
- From your templates. If pages are rendered from data, add a Markdown renderer for the same data. Our own pricing is published this way, at /pricing.md, generated from the same plan data as the pricing page.
- From the rendered page. For everything else, convert the rendered HTML with a converter that keeps the main content only — and check the output once, since navigation and consent text are what usually leak in.
Whichever you choose, keep the Markdown and the page in step. A Markdown twin that lags the page is worse than none, because the tools that read it trust it.
Writing one
How to write a good llms.txt
- 1Write the summary for a reader who knows nothing. What it is, for whom, and the one or two facts that most often get misunderstood. This paragraph is what gets quoted.
- 2Link to fewer pages than you think. Ten excellent links beat two hundred. If a page wouldn't help a newcomer understand the product, it belongs under Optional or nowhere.
- 3Link to Markdown where you can. A link to an HTML page makes the tool do the conversion; a
.mdversion hands it clean text. - 4Put the numbers in, and keep them true. Prices, limits and supported formats are the facts people ask models about. Generate them from the same source as your site.
- 5Test it with a model. Give an assistant only your llms.txt and ask it the ten questions customers ask. Wrong answers show what's missing.
For pages that don't have a Markdown version yet, a converter that renders the page and keeps only the main content can produce one — the URL to Markdown tool shows what that output looks like for any page, and the /markdown endpoint does it from code, for 1 credit a page and without a model.
FAQ
Frequently asked questions
What is llms.txt?
Does llms.txt help with SEO or Google AI Overviews?
Who actually reads llms.txt?
What is the difference between llms.txt and robots.txt?
What is llms-full.txt?
Try it yourself
Free tools for this
No signup — run these on a real page right now, then call the same endpoint from your code.
Put this into practice.
Each of the eight kinds of data URLpipe returns has a free, no-signup tool — try the ideas from this guide on a real page, then grab an API key to run them from your code. 1,000 credits a month, no card.