AI Search
llms.txt Explained: What It Does and Doesn’t Do
llms.txt is a proposal, not a search standard. Here’s what that means in practice.
llms.txt is a proposed file, published at llmstxt.org, that a site can place at /llms.txt to give large language models a concise, markdown-formatted guide to its most useful content.
It’s often confused with robots.txt and presented as a requirement for AI visibility. Google’s own documentation says otherwise for its AI features.
Short answer: llms.txt is a proposed markdown file at a site’s root that summarizes the site and links to key pages for large language models. It doesn’t control crawling or indexing, and it isn’t a ranking signal. Google says no new machine-readable files or AI text files are needed to appear in AI Overviews or AI Mode. Adding one is low-risk, but it shouldn’t replace the basics: crawlable, indexable, clearly written pages.
Key Takeaways
- llms.txt is a proposal, not an adopted search standard.
- It doesn’t block or allow crawling; that’s robots.txt.
- Google says no AI text files are needed for its AI features.
- It’s low-risk to add but not a substitute for good pages.
- Don’t pay for “llms.txt optimization” as a visibility fix.
llms.txt vs robots.txt
| robots.txt | llms.txt | |
|---|---|---|
| Purpose | Tells crawlers which URLs they may request | Summarizes the site for language models |
| Status | Established standard; Google documents its support | Proposal |
| Effect on Google Search | Controls Googlebot crawling | Google says AI text files aren’t needed |
| Format | Plain-text rules | Markdown |
What Google says
Google’s guidance on AI features says there are no additional requirements to appear in AI Overviews or AI Mode, and that you don’t need to create new machine-readable files, AI text files or special schema. Pages need to be indexed and eligible to show with a snippet.
Should you add one?
- It does no harm if kept accurate and short.
- It may help tools that choose to read it; adoption varies by tool.
- It won’t fix pages that are blocked, thin or unclear.
- Keep it consistent with what your pages actually say.
Higher-impact work: making content easier to cite and choosing AI crawler access.
Frequently asked questions
Does Google use llms.txt?
Google says AI text files aren’t needed to appear in its AI features.
Can llms.txt block AI crawlers?
No. Crawler access is controlled with robots.txt and each crawler’s documented user agent.
Is llms.txt required for ChatGPT visibility?
OpenAI’s crawler documentation focuses on allowing OAI-SearchBot in robots.txt for ChatGPT search visibility.
Want your content ready for AI search?
We review how your pages, entities and structured data hold up in AI-powered search, and fix the SEO foundations those features rely on.
Sources
- Google Search Central: AI features and your website
- llmstxt.org: The /llms.txt file proposal
- OpenAI: Overview of OpenAI crawlers
Last reviewed October 14, 2026.
Table of Contents
Preparing for AI Search?
We strengthen the SEO foundations AI features draw on: indexable, clear, trustworthy pages.
Related Guides
Free SEO Audit
A manual on-page and technical SEO review of your website, with prioritized fixes.