llms.txt
llms.txt is the AEO equivalent of `robots.txt` plus a sitemap, written for LLMs. It's promising, contested, and cheap to implement, which is why most teams are adding it even without proof it helps.
Strong signal and real results. Worth committing a pilot to.
llms.txt·Schema
What It Is
llms.txt is a community-driven proposal from Jeremy Howard at Answer.AI for a Markdown file hosted at the root of your domain (`/llms.txt`). It gives large language models a clean, structured summary of your site's most important content, with links to canonical sources. A companion format, `llms-full.txt`, includes the full content of those sources concatenated in a single file. Mintlify co-developed `llms-full.txt` with Anthropic and rolled it out to thousands of Mintlify-hosted docs sites in November 2024.
Why It Matters
The case for llms.txt is straightforward: LLMs that crawl your site face the same content-vs-chrome problem human search engines once did, and a curated Markdown summary is faster and cheaper to ingest than scraping your full HTML. Anthropic, Cloudflare, Vercel, Cursor, Coinbase, Pinecone, and Windsurf all serve llms.txt files. Vercel reports that around 10% of their signups now come from ChatGPT, which they attribute partly to GEO (generative engine optimisation) work that includes llms.txt.
The case against is also worth understanding. Google has stated llms.txt is not a ranking signal and has no recommended use in Search. Adoption among major non-tech brands remains low, and direct evidence that LLMs prefer llms.txt content over normal HTML is thin. For most AEO buyers in 2026, llms.txt is a cheap-to-implement bet rather than a proven win. The cost of being wrong is low; the cost of missing out if it does become a citation signal is higher.
Key Developments
- 2026: Adoption broadens in developer-tooling and SaaS. Skepticism crystallises among SEO practitioners, with Google publicly stating llms.txt is not a ranking factor.
- Nov 2024: Mintlify and Anthropic co-developed `llms-full.txt`. Rolled out across thousands of Mintlify-hosted docs sites including Anthropic, Cursor, Coinbase, Pinecone, and Windsurf.
- 2024: Jeremy Howard published the original llms.txt proposal at llmstxt.org.
What to Watch
Watch whether any major AI engine publicly states it uses llms.txt for retrieval or citation. None has, as of mid-2026. Track adoption beyond developer tooling, especially among consumer brands and publishers, which would signal that the format is becoming load-bearing rather than a developer convention. Watch Cloudflare's posture. If Cloudflare bundles llms.txt generation into its AI bot management tools, adoption could jump significantly.
Strengths
- Cheap to implement: A single Markdown file at the domain root. Effort is minimal for the upside.
- Supported by major AI labs: Anthropic publicly serves one. Cloudflare, Vercel, and the Mintlify ecosystem give it a credible early adopter base.
- Format is sensible for LLMs: Markdown is what LLMs are trained on. Structured summary plus canonical links is genuinely easier to consume than raw HTML.
- Future-proofs for citation: If LLMs start preferring llms.txt content over scraped HTML, early adopters benefit. Cost of being early is low.
Considerations
- Not a ranking signal in Search: Google has publicly stated llms.txt has no recommended use in Search.
- Thin evidence of citation impact: No major AI engine has confirmed that llms.txt content is preferred over normal HTML for retrieval.
- Maintenance overhead: Keeping llms.txt in sync with site content is a real ongoing cost, especially for content-heavy sites.
- Adoption skewed to dev tooling: Most adopters today are SaaS, dev-tools, and AI labs. Consumer-brand and publisher adoption is rare.
Articles
Background on the llms-full.txt format co-developed with Anthropic.
Examples of how leading tech companies structure their llms.txt files.
Critical analysis from a major SEO platform on whether llms.txt matters.
llms.txt· AI crawler controls· Content freshness signals· Entity authority· E-E-A-T signals· Schema.org structured data