AI SEO Glossary
What is llms.txt?
TL;DR — llms.txt is a community-proposed Markdown file at /llms.txt that gives large language models a curated, machine-readable map of a site's most important content. Proposed by Jeremy Howard of Answer.AI in September 2024 and increasingly adopted across the AI tooling ecosystem.
Definition & scope
llms.txt is a community-proposed text-file standard that lives at /llms.txton a website and provides large language models with a curated, machine-readable map of the site's most important content. Proposed by Jeremy Howard of Answer.AI in September 2024. The file is written in Markdown, opens with the site name (H1) and a short blockquote summary, and lists key sections — about, product categories, reference content, policies — as labeled links.
Distinct from robots.txt (directive — what crawlers may access) and sitemap.xml (comprehensive — every indexable URL); llms.txt is editorial — a curated subset that tells an LLM “here are the 20-40 pages that explain who we are.” Adopted across Anthropic Claude, Cursor, MCP tooling, and increasingly the major engines through 2025-2026. Full spec, sample file, and implementation guide on /llms-txt-implementation.
Where you'll encounter it
On the websites of AI-forward companies (Anthropic ships one, Cursor ships one, most MCP-tooling vendors ship one). Inside Claude Projects and Computer Use sessions, where Claude can be prompted to fetch and parse a target site's llms.txt to get oriented. In agency proposals as a low-cost foundation deliverable.
We ship llms.txtas a foundation deliverable on every AI-SEO retainer. It's cheap to produce, durable (changes infrequently after first publication), and increases the surface area on which our other AEO/GEO work pays dividends.
Related terms
- MCP — the protocol many llms.txt-aware tools also speak.
- AEO — the citation-targeted optimization llms.txt supports.
- GPTBot / ClaudeBot — crawlers that complement llms.txt with full-site coverage.
- Structured data — the per-page machine-readable layer that complements the site-level llms.txt map.
Related services
- llms.txt Implementation — our ship-it-this-sprint deliverable.
- AI SEO Services — the parent retainer.
FAQ
Who proposed llms.txt?
Jeremy Howard of Answer.AI proposed the standard in September 2024. The full spec lives at llmstxt.org with community implementations across the developer-tooling and AI ecosystems.
How is llms.txt different from robots.txt and sitemap.xml?
robots.txt is directive (what crawlers may or may not access). sitemap.xml is comprehensive (every indexable URL). llms.txt is editorial — a curated subset that tells an LLM “here are the 20-40 pages that explain who we are.” All three are complementary.
What does the llms.txt file look like?
Markdown. Starts with an H1 site name, then a blockquote one-sentence summary, then sections (about, services, reference, policies) as labeled link lists. Sample skeletons are published at llmstxt.org; we ship a customized version per client.
Do the major AI engines read llms.txt?
Anthropic Claude, Cursor, and a broad MCP tooling ecosystem actively consume llms.txt as of 2025-2026. Major engine adoption (OpenAI, Google) is increasing but not yet universal. The cost to publish is low enough that the standard is worth shipping today.
Where should I publish llms.txt?
At /llms.txt on your site root — same convention as robots.txt. Some sites also publish /llms-full.txt with the actual content concatenated, so an LLM can ingest the full corpus in one fetch. See /llms-txt-implementation.
Will llms.txt help my AEO citation share?
Indirectly. llms.txt isn't a citation signal on its own — it's a discoverability and grounding aid. But making your most cite-worthy content easier for LLMs to find compounds with the rest of your AEO signal. Low-cost foundation work.
Ship your llms.txt this sprint.
Cheap foundation deliverable, lasting AEO surface lift. Call 888-982-8269 to scope.