Does llms.txt Work? What the Data Actually Shows (Plus a Template)

Here's a sentence you'll see in most articles about llms.txt: "This simple file tells AI crawlers everything they need to know about your website."

Here's a sentence you won't: Google's head of search explicitly stated that no AI system currently reads it.

Both are true. That tension is the whole story, and it's more interesting than either the hype or the dismissal.

AI referral traffic grew 796% between January 2024 and December 2025, according to Adobe Analytics tracking over a trillion web visits. Businesses are scrambling for anything that improves their visibility in that channel. llms.txt arrived in September 2024, immediately got overhyped, then got dismissively debunked, and now sits in an uncomfortable middle ground most articles won't occupy honestly.

We added one to the Untap Web site. Here's what the file actually is, what the research says about whether it works, and how to create one in the next 15 minutes, copy-paste template included.

Key Takeaways

• No major AI platform has officially committed to reading llms.txt; Google explicitly rejected the standard (Google Search Central, 2025)
• A study of 300,000 domains found zero correlation between having llms.txt and AI citation frequency (SE Ranking, Nov 2025)
• Only 0.3% of the top 1,000 websites have one, putting you ahead of 99.7% of the competition today (Rankability, June 2025)
• robots.txt took 28 years to become a formal standard; early adopters still dominated search indexing from day one

What Is llms.txt, and Where Did It Come From?

Jeremy Howard, co-founder of Answer.AI and one of the researchers behind the fast.ai deep learning library, published the llms.txt specification on September 3, 2024 at llmstxt.org. The concept is straightforward: just as robots.txt tells search engine crawlers where to go and where to stay out, llms.txt tells AI language models what your website is about and which pages deserve attention.

The file lives at yourwebsite.com/llms.txt. It uses basic markdown: a title, a short description, and a list of links to your important pages with one-sentence summaries. No JSON. No schema syntax. No developer required if you have FTP access.

There's also an optional companion: llms-full.txt. Where llms.txt is the index, a curated menu of links, llms-full.txt is the full meal: all the content from your key pages concatenated into one file. That lets an AI read everything in a single request without following individual links. For a service business with five to ten pages, creating both files takes under an hour (HitlSEO, 2025).

See our complete guide to what AEO is and why it matters for the broader context on AI search visibility.

Does llms.txt Actually Work? Here's What the Data Says

In November 2025, SE Ranking published the largest study on llms.txt to date: an analysis of approximately 300,000 domains that found no meaningful statistical correlation between having an llms.txt file and AI citation frequency (Search Engine Journal, November 2025). Two months later, Search Engine Land tracked 10 websites across five industries for 90 days before and after adding llms.txt. Eight of ten saw no measurable change in AI-referred traffic. The two that saw improvement attributed the gains to PR coverage and new content published during the same window — not the file.

The most quotable verdict came from Google's John Mueller, who stated directly that no AI system currently uses llms.txt. Published server log data backed him up: GPTBot, ClaudeBot, and PerplexityBot were not consistently fetching llms.txt files over a tracked 10-week window.

So as of early 2026: llms.txt does not demonstrably improve AI citations. Full stop.

The honest version of this post doesn't bury that finding. Most articles either skip it or hide it in a footnote because they want to sell you on a new tactic. The research is clear, and you deserve to know it before you spend any time on this.

Then why does Untap Web have one? And why are we recommending you add one?

Why You Should Add One Anyway: The robots.txt Argument

In February 1994, a web developer named Martijn Koster posted a proposal to a mailing list: a plain text file at every website's root directory, telling crawlers what they could and couldn't index. He called it robots.txt.

Four months later, AltaVista, Lycos, and WebCrawler had voluntarily adopted it. No RFC. No official standard. No search engine formally requiring it. Just a good idea, a simple format, and early movers who shaped how crawlers learned to read the web before anyone asked them to.

The formal IETF standard, RFC 9309, arrived in September 2022. Twenty-eight years after the original proposal.

The websites that added robots.txt in 1994 didn't wait for a blessing that took nearly three decades to arrive. They planted a flag in a developing standard, and that early signal shaped how their sites were indexed for the entire era of search that followed.

llms.txt was proposed in September 2024. We're somewhere between "Martijn Koster just posted his mailing list message" and "AltaVista voluntarily adopted it." The standard is 18 months old.

In 2025, Perplexity processed approximately 780 million search queries per month, up 239% from the prior year, according to Backlinko's analysis of public platform data. ChatGPT processes over 2 billion queries per day. AI referral traffic grew 796% across the web in a single year. This channel is growing faster than any prior search channel in history. Spending 15 minutes to plant a flag costs nothing. The downside risk is exactly zero.

When we added llms.txt to the Untap Web site as part of a broader AEO push in early 2026, Perplexity cited our site the following morning on a local web design query. Did the file cause it? Probably not directly, as the research is clear that it doesn't work that way yet. But it was one signal in a pattern of AEO work, and the total implementation took under an hour. We've since added it to every client site we build. The file is there, it costs nothing, and when the standard matures we won't be scrambling to catch up.

The Exact Template: Copy and Adapt This

The official specification at llmstxt.org is minimal by design. Here's a working template for a local service business, annotated:

# Your Business Name: Service Category

> One to two sentences: what you do, who you serve, where you operate.
> "Web design and local SEO for HVAC, plumbing, and landscaping companies
> in Missouri and Illinois" beats "digital marketing services."

## Services

- [Service Name](/services/service-slug): One sentence on this service and who it's for.
- [Second Service](/services/second-slug): Focus on the outcome for the customer.
- [Third Service](/services/third-slug): Keep every description under 15 words.

## Key Pages

- [About](/about): Who runs this business, background, years of experience.
- [Compare Plans](/compare): Side-by-side service tiers and pricing.
- [Contact](/contact): How to reach us and request a free estimate.

## Blog

- [Post Title](/blog/slug): One-sentence summary of what this article answers.
- [Another Post](/blog/slug): Keep summaries tight, as context windows are limited.

## Optional

- [Privacy Policy](/privacy): Legal privacy disclosures.
- [Terms of Service](/terms): Terms governing use of our services.

A few rules worth knowing:

• The # H1 is the only required element. Everything else is optional.
• Write the blockquote as if briefing an AI that has never heard of your business. Specific details beat generic claims at every point.
• Keep link descriptions under 15 words. Long descriptions eat context window space that AI tools could spend reading your actual pages.
• The ## Optional section is a signal to AI crawlers: these pages can be skipped when context is tight.

How to Add It to Your Website in 5 Steps

This takes 15 minutes. No developer required if you have FTP or file manager access to your hosting.

Step 1: Write the file. Copy the template above into any plain text editor: Notepad, TextEdit, VS Code. Swap in your real business name, services, and actual page URLs.

Step 2: Save as plain text. File name: llms.txt. Not llms.html, not llms.docx. Plain .txt so AI parsers read it directly.

Step 3: Upload to your website root. The file belongs at yourdomain.com/llms.txt, the same location as robots.txt and sitemap.xml. On WordPress, upload via your hosting file manager to public_html/. On Next.js, drop it in your public/ directory.

Step 4: Verify. Visit yourdomain.com/llms.txt in your browser. You should see your plain text, not a 404 or an HTML page.

Step 5: Create llms-full.txt (optional but worth doing). Open each key service page, copy the main body content as plain text, and paste everything into a second file named llms-full.txt. Upload to the same root directory. This file can be large, and that's expected and fine.

For reference, here's what the Untap Web llms.txt looks like in practice:

# Untap Web: Web Design & Local SEO for Small Businesses

> Subscription-based web design, local SEO, and AEO for service businesses
> in Missouri, Illinois, and the Midwest. No setup fees. Cancel anytime.

## Services

- [Web Design](/services/web-design): Subscription websites built for speed, SEO, and lead generation.
- [Local SEO](/services/local-seo): Google Maps and local search optimization for service businesses.
- [AEO](/services/aeo): Answer engine optimization to get cited in ChatGPT, Claude, and Perplexity.

## Key Pages

- [Compare Plans](/compare): Stand Up ($249/mo) vs Growth ($649/mo), full feature breakdown.
- [About](/about): Founded by Chris Melson in 2026. Background in web performance and local search.
- [Showroom](/showroom): Before-and-after examples with Lighthouse scores and traffic results.

## Blog

- [What Is AEO?](/blog/what-is-aeo-and-why-it-matters): Answer engine optimization explained for small business owners.
- [How to Get Cited by ChatGPT](/blog/how-to-get-cited-by-chatgpt-claude-and-perplexity): Five practical steps with statistics and platform-specific guidance.

For the broader strategic picture, see how we structure AEO alongside local SEO for small businesses.

How to Check Whether AI Crawlers Are Visiting the File

Honest expectation first: in early 2026, you probably won't see much. Published log data confirms major AI bots are not consistently fetching llms.txt files. But setting up monitoring now means you'll catch the moment that changes.

The bots to watch for in your server access logs:

Bot	Operator	String to search
GPTBot	OpenAI	GPTBot
OAI-SearchBot	OpenAI	OAI-SearchBot
ClaudeBot	Anthropic	ClaudeBot
PerplexityBot	Perplexity	PerplexityBot
Google-Extended	Google	Google-Extended

If you have SSH access to your server logs, this command pulls all AI bot activity:

grep -i "GPTBot\|ClaudeBot\|PerplexityBot\|OAI-SearchBot" /var/log/nginx/access.log

Look specifically for any of these bots requesting /llms.txt. If your hosting platform has a bot traffic dashboard, use it to filter by user agent. Screaming Frog's Log File Analyser can parse downloaded access logs if you prefer a GUI.

Set a quarterly reminder to run this check. When logs show these bots fetching the file consistently, the standard has arrived. You'll already be ahead.

After adding llms.txt to the Untap Web site, we checked our server logs weekly for the first month. We saw ClaudeBot visits to our general pages, normal crawl behavior, but no specific requests to /llms.txt. That matches what the research predicts. It didn't change our calculation. The file stays.

If you want all of this built into your site from day one, our AEO service covers schema, FAQ structure, llms.txt, and answer-first content together.

---

The file takes 15 minutes. The research says it doesn't work yet. You should still have one.

Not because you're chasing hype, but because the cost is nothing, the upside compounds over time, and you'd be an early adopter in a channel growing 796% per year. robots.txt wasn't required in 1994 either. The websites that added it anyway shaped how the entire era of search worked.

If you want AEO built into your site from the ground up (answer-first content, FAQ schema, llms.txt, and the rest), our AEO service handles all of it.