Updated 17 July 2026. We originally published this post recommending llms.txt. Since then Google has stated plainly that it ignores the file, and the first large-scale data on who actually reads it has landed. We have rewritten this post rather than quietly deleting it, because we would rather be the agency that corrects itself in public than the one still selling last year’s theory.
The short answer
An llms.txt file will not get your business recommended by ChatGPT, Google AI Overviews or Perplexity. It is harmless, it costs nothing, and there is a narrow case for it. But if anyone tells you it is how you get found by AI, the evidence is not on their side.
What llms.txt is
llms.txt is a plain-text file at the root of your site (yoursite.co.za/llms.txt) that offers AI tools a clean, curated map of your content — what your business does, which pages matter, in Markdown rather than cluttered HTML. It is a genuinely sensible idea. The theory was that it would become the robots.txt of the AI era.
The theory has not survived contact with the data.
What Google actually says
Google’s Search Central AI optimisation guide, updated 10 July 2026, is unambiguous:
“You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn’t use them.”
Google adds that publishing one “will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them.” Google’s John Mueller has separately described llms.txt as “not done for search” — closer to a temporary crutch for AI coding tools than a search feature. Google’s guide is public; you can read it yourself.
What the data says
In May 2026, Ahrefs analysed llms.txt traffic across 137,210 domains. The findings:
- 97% of llms.txt files received zero requests in the entire month — no bots, no humans.
- AI search crawlers (OAI-SearchBot, PerplexityBot, Claude’s search crawler) accounted for 1.1% of the requests that did arrive.
- Slackbot fetched llms.txt more often than PerplexityBot did.
- AI bots made zero requests for llms.txt files that didn’t exist. They never go looking.
The bots that do read llms.txt are mostly coding agents and training crawlers — not the retrieval systems that decide which business gets named in an answer.
So is there any reason to have one?
A narrow one, and we will be honest about how narrow:
- AI coding agents do read it. If you publish developer documentation, it has a real audience.
- It costs nothing and breaks nothing. Ten minutes, no downside.
- It might matter later. If AI agents come to mediate more of search, the file could find a purpose through that layer. Nobody knows.
- One caution: the single largest crawler of llms.txt files in the Ahrefs data identifies itself as a prompt-injection survey. Someone is systematically probing these files as an attack surface. Don’t put anything in yours you wouldn’t put on a public page.
Add one if you like. Just don’t pay anyone much for it, and don’t expect it to move your AI visibility.
What actually gets you recommended by AI
Here is the part worth your attention. AI answer engines are retrieval systems with a language model attached at the end. The model doesn’t decide who gets recommended — the index does. Google confirms its AI features are “rooted in our core Search ranking and quality systems” and use retrieval-augmented generation against the normal Search index.
That means the things that get you cited are unglamorous:
- Be indexed and snippet-eligible. For Google’s AI features this is a hard gate — a page must be indexed and allowed to show a snippet to be cited at all. A stray
nosnippettag silently disqualifies you. - Don’t block the retrieval bots — in robots.txt and at your firewall or CDN. This is the most common real failure we find.
- Rank. 76.1% of pages cited in Google’s AI Overviews already rank in Google’s top 10. Ranking and AI citation are not separate games.
- Stay fresh. Across 17 million citations, AI-cited pages were on average 25.7% newer than standard organic results.
- Say something only you can say. Google’s guidance explicitly rewards first-hand experience over “a summary of existing content” that “could easily be produced by a generative AI model.”
- Be a clear entity. Correct structured data that matches your visible content, plus a properly maintained Google Business Profile.
The reassuring bit for South African businesses
AI Overviews appear on only 7.9% of local searches, against a 20.5% baseline across all searches. If you’re a plumber in Table View or a dentist in Sea Point, AI is currently reshaping far less of your world than the headlines suggest. Your Google Business Profile still does the heavy lifting.
And here’s the opportunity: 86% of the most-cited sources are not shared between ChatGPT, Perplexity and Google AI Overviews. “Optimising for AI” isn’t one target — it’s several. Perplexity in particular pulls from a broader regional pool, which favours smaller local brands over the usual global incumbents.
How we work
We will still add an llms.txt file to your site if you want one — it takes ten minutes and it does no harm. But we won’t charge you a premium for it or pretend it’s the reason AI will find you. Our AI Search Visibility work is built on the things the evidence actually supports: being crawlable, indexed, correctly marked up, genuinely useful, and unmistakably the best answer to a real question.
If you’d like to know where your site currently stands, our free website scan checks the real signals — and shows you the raw numbers it measured, so you can verify every one.