How to create an llms.txt file that is useful, concise, and safe
An llms.txt file is a public Markdown file placed at the root of a website. The goal is simple: give language models and AI agents a compact map of the site, the most important public pages, and any interpretation notes that prevent confusion.
llms.txt is an emerging convention, not a guaranteed ranking factor and not a replacement for normal SEO, robots.txt, or access control.
Recommended format
The proposal at llmstxt.org uses Markdown. The only required part is an H1 title, but a useful file normally has a short summary, notes, a core page list, and an optional page list.
# Site name > One short summary of what this website does and who it serves. Important notes: - Use these links as the canonical public sources. - Do not include private pages, unpublished claims, or customer data. ## Core pages - [Product overview](https://example.com/product): Main public product page - [Docs](https://example.com/docs): Documentation and setup guide ## Optional - [Blog](https://example.com/blog): Secondary educational articles
What to include
Include pages that help an AI system answer a user's question accurately: product pages, documentation, pricing, policies, glossary pages, support pages, and high-quality original guides. Every link should already be public and safe to be read by humans and crawlers.
Write descriptions that reduce ambiguity. For example, instead of saying "pricing page", say "current self-serve subscription plans and billing limits". That extra context helps a reader understand why the page matters.
What to avoid
Do not include customer data, private documents, staging URLs, admin paths, API keys, hidden discount URLs, unreleased product claims, or anything you would not want publicly indexed. The file is public by design.
Do not stuff the file with keyword lists. If it reads like a doorway page or meta keywords tag, it is doing the wrong job. The useful version is a curated map, not a ranking trick.
Validation checklist
- The file loads at
https://yourdomain.com/llms.txt. - The first line is a single H1 with the site or project name.
- The summary is accurate and not promotional hype.
- Every linked URL returns a live 200 response or a deliberate redirect.
- Descriptions are short and explain the purpose of each page.
- The optional section contains secondary material only.
How this fits SEO
Google's guidance for AI features says the core SEO fundamentals still matter: crawlability, internal links, useful visible text, page experience, and structured data that matches the page. So treat llms.txt as an extra clarity layer, not as a shortcut around building useful pages.