llms.txt: Is It Worth Adding to Your Website?
llms.txt is a plain file meant to guide AI tools around your site. What the data says about whether it works, and when it is still worth the ten minutes.
Probably not as a ranking or visibility tactic, but it costs almost nothing, so it is harmless to add. llms.txt is a proposed Markdown file that summarises your site for AI tools, yet Google has said it does not use it, and the large studies so far found no clear link between having the file and being cited in AI answers. Treat it as optional housekeeping, not a strategy.
That answer will disappoint anyone who read that llms.txt is "the new robots.txt". It is not, and the difference matters if you are deciding where to spend a limited budget. Below: what the file is, what the evidence shows, the cases where it genuinely helps, and what to do first instead.
What is llms.txt, exactly?
llms.txt was proposed in September 2024 by Jeremy Howard of Answer.AI. The idea is simple: put a Markdown file at yourdomain.com/llms.txt containing the name of your site, a short summary, and a curated list of links to your most useful pages, each with a one-line description. A language model with a limited context window could read that single file instead of crawling dozens of heavy HTML pages.
It is a community proposal, not an internet standard. Nobody ratified it, and no search engine or AI company is obliged to read it. That is the key difference from robots.txt, which is defined in a public standard and which well-behaved crawlers honour. robots.txt tells bots what they may fetch. llms.txt only offers a suggested reading list.
Do AI systems actually read it?
The evidence so far says: rarely, and not in a way you can measure as a benefit.
On the Google side, John Mueller has said that no AI system currently uses llms.txt and has compared it to the old keywords meta tag, a signal that everyone could add and nobody could trust. Gary Illyes is reported to have said in mid-2025 that Google does not support the file and has no plans to. Google's own guidance on AI features in Search says nothing special is needed beyond good SEO.
On the data side, two large analyses point the same way. SE Ranking looked at about 300,000 domains, found roughly one in ten had an llms.txt file, and concluded that having it showed no clear effect on how often a domain was cited in AI answers. Ahrefs analysed 137,000 sites and reported that the large majority of llms.txt files received no requests from AI crawlers at all. Both are SEO-vendor studies with their own methods and limits, and the picture could change, but nobody has produced solid evidence of a visibility gain.
The practical reading is simple. Nobody can promise you that the file brings visitors, mentions or citations, so be sceptical of any agency that sells llms.txt as a standalone service or a guaranteed route into ChatGPT answers. At best it is a small convenience; at worst it is a distraction from work that matters more.
One honest caveat: absence of evidence is not proof it never helps. Some AI tools and coding assistants do fetch the file when a user points them at it, and that is where it is actually used.
When is llms.txt still worth adding?
Three situations make the ten minutes reasonable.
**You publish technical documentation.** Developer docs, API references and product manuals are the original use case. Coding assistants and agents can pull a clean index of your docs, and many documentation platforms generate the file automatically.
**Your site is large or confusing.** If you have hundreds of pages, a curated list of the twenty that matter is a useful summary, for people as much as for machines. Writing it forces you to decide what your site is actually about.
**You like being early and it costs nothing.** If your site is built so that the file is generated from the same content as the pages, maintaining it is trivial. It is a reasonable bet with no downside, as long as you do not expect a result.
If you run a typical small business site with ten or twenty pages, the file will add little, because a clear navigation, a sitemap and well-structured pages already do the same job better.
What should you do before llms.txt?
The things that demonstrably affect whether AI products can find and quote you are the unglamorous ones. Make pages fast and crawlable. Answer real questions directly near the top of the page. Use accurate structured data so machines can identify who you are and what you sell. Keep a current sitemap. And decide deliberately which AI bots may crawl you in robots.txt, because blocking the wrong one can remove you from answers.
These steps help classic search and AI search at the same time, which is why we would always do them first. A site that is slow, thin or badly structured will not be rescued by a Markdown index.
How do you write a useful llms.txt?
If you decide to add one, keep it short and honest. The format is plain Markdown: a title line with your site name, a one- or two-sentence summary in a quote block, then sections with bullet lists of links, each followed by a colon and a short description of the page.
Include only pages you want understood: your main services, pricing, contact, a few cornerstone articles. Describe each in plain words, not marketing language. Leave out thank-you pages, tag archives and anything outdated. Publish it at the root of the domain, as text, and check that it loads at yourdomain.com/llms.txt without a redirect.
Then treat it like any other file: review it when you add or remove important pages. An out-of-date index that points to deleted URLs is worse than none.
Where do we come in?
On sites we build, files such as the sitemap, robots.txt and structured data are generated from the same content as the pages, so adding or updating an llms.txt is a small change rather than a manual chore. If you would like an opinion on what your site actually needs for AI visibility, see our web design service or the pricing and tell us what you publish. Black Friday prices continue through Christmas: −50% on every package, valid until 31 January 2027.
Frequently asked questions
- Does llms.txt help me rank in Google?
- No. Google has said it does not use llms.txt for Search, and its AI features rely on the same crawling and quality signals as normal results. Adding the file will not move your rankings.
- Is llms.txt the same as robots.txt?
- No. robots.txt is a standard that tells crawlers what they may or may not fetch. llms.txt is only a proposed reading list in Markdown; it blocks nothing and no AI company is required to follow it.
- Can llms.txt hurt my site?
- Not in itself, but an outdated file that lists deleted pages or contradicts your site can mislead the tools that do read it. Keep it short and review it when your key pages change.
- Where do I put the file?
- At the root of your domain, so it opens at yourdomain.com/llms.txt, served as plain text or Markdown. It does not belong in a subfolder, and it does not replace your sitemap.
Black Friday prices continue through Christmas: −50% on every package.
Have a project in mind?Let’s build something memorable.
Start a project