llms.txt
llms.txt is an emerging standard for a plain-text file at the root of a website that summarises the site for AI assistants — analogous to robots.txt for crawlers or sitemap.xml for search engines.
Why it matters
AI assistants ingest a limited amount of content per source. llms.txt lets a brand hand the AI a curated digest of its highest-signal pages — overview, AEO offering, pricing, FAQs, glossary — so the AI doesn't have to crawl semi-randomly to figure out what the site is about.
What it does
An llms.txt file lives at the root of a domain (e.g. example.com/llms.txt) and provides a structured markdown summary of the site, with links to canonical pages grouped by section. The companion llms-full.txt includes the rendered text of those pages so an AI can ingest a complete picture without crawling.
The format
Per the llmstxt.org spec, the file uses markdown:
- H1 — the project / site name.
- Blockquote summary — one paragraph describing the site.
- H2 sections — grouped lists of links with one-line descriptions.
Why it matters for AEO
AI assistants have an attention budget. If they have to crawl twelve pages to learn what your site is about, they spend that budget on HTTP fetches and parsing instead of comprehension. A good llms.txt hands them a curated digest in one request — and a great llms-full.txt lets them answer 'what does this company do' / 'how much does it cost' without ever touching the rest of the site.
Adoption
The standard is still emerging but already adopted by Anthropic docs, Vercel, Cloudflare, LangChain, and a growing list of AI-native companies. For brands serious about AEO, shipping an llms.txt is table stakes — the absence is a small but real reputational signal.
Related terms
