Omni Care logo Omni Care

AI readability · 8 min read

A File Does Not Make Your Website AI-Readable

Many website teams are being told to add an llms.txt file.

The pitch sounds neat. Put one file at the root of the site, list the important pages, and AI tools will understand the business more easily.

That may become useful for some systems over time, especially for documentation-heavy sites. It is not a reliable shortcut to AI visibility today.

Google's own Search Central guidance says websites do not need new AI text files, special machine-readable files, or special markup to appear in Google Search, including its generative AI features. Google Search does not treat llms.txt as a special ranking or inclusion signal.

Ahrefs also looked at server-log data from 137,210 domains in May 2026. Among the sites in that dataset with valid llms.txt files, 97% received no requests for the file at all during the month. No bots. No humans. No AI retrieval traffic.

That does not mean "do nothing."

It means the work is not the file. The work is the layer around the website that lets machines understand the site without guessing, and lets a human team keep that layer correct after the site changes.

The Real Question

The useful question is not:

Does this website have llms.txt?

The useful question is:

Can an AI assistant reach the right pages, understand what the business offers, answer common buyer questions from on-domain sources, and avoid stale or misleading information when the website changes?

That is a technical layer, not a single artifact.

Layer 1: Structured Data On The Pages That Matter

A homepage with plain HTML can look fine to a human visitor and still be ambiguous to a machine.

Structured data helps identify what a page represents: the organization, the product, the service, the article, the location, the FAQ, the event, the offer, or the breadcrumb path.

This does not require a special AI schema. Google's current guidance says there is no special schema.org markup required for generative AI search. But structured data still matters as part of the normal search and rich-result layer, and it reduces avoidable ambiguity.

For a practical first pass, check:

  • Does the homepage describe the organization clearly?
  • Do service and product pages carry the right entity type?
  • Do FAQ sections use FAQPage only when the page actually contains FAQ content?
  • Do location, event, menu, booking, or product pages carry the specific schema they need?
  • Is the structured data updated when page content changes?

The point is not to stuff schema everywhere. The point is to make the important pages machine-readable without lying about what is on them.

Layer 2: Access Decisions For Crawlers And Agents

A site can accidentally make itself hard to inspect.

Sometimes the problem is a missing sitemap. Sometimes an important help section sits on a separate subdomain. Sometimes robots.txt blocks paths that a crawler or browser-based agent would need to inspect. Sometimes every missing machine file returns an HTML app fallback with HTTP 200, which looks successful to a shallow check but is not a real machine-readable endpoint.

The practical check is simple:

  • Can important pages be fetched without a login?
  • Does robots.txt point to the correct sitemap?
  • Are useful crawlers blocked deliberately, or by accident?
  • Do missing files return honest 404s instead of fake app pages?
  • Do server logs show any useful AI or agent traffic already visiting the site?

Server logs matter because they show what is actually happening. Ahrefs' study is a good warning: a file can exist, and almost nobody may request it.

Layer 3: Answer-First Content

AI assistants need answerable material.

Most business websites have persuasive copy. Fewer have direct answers to the questions a buyer or customer would ask:

  • What does this service do?
  • Who is it for?
  • How does pricing or booking work?
  • What happens after I submit a request?
  • What areas, stores, products, plans, or use cases are covered?
  • What should I prepare before talking to the team?

If those answers live only in PDFs, sales decks, chat widgets, or off-domain help centers, the main site may not carry enough source material for an assistant to answer confidently.

Answer-first content does not mean writing for robots. It means giving people and machines the same clear answers on pages the business owns.

Layer 4: Machine-Readable Support Files, Kept In Their Place

This is where llms.txt belongs: one support file, not the strategy.

If a business already has a maintained sitemap, structured data, clear service pages, and answer-first FAQ content, a well-maintained llms.txt can act as an extra index. It can point AI tools toward the best public pages and reduce wasted crawling for tools that choose to read it.

But it should be treated like code:

  • generated from current source pages where possible;
  • reviewed when page structure changes;
  • linked from real places if a tool needs to discover it;
  • monitored in server logs;
  • removed or corrected if it drifts away from the live site.

An outdated machine-readable file is worse than a missing one. It gives machines a cleaner path to stale information.

Other agent-facing formats may matter later. WebMCP and related agent-on-site ideas are worth watching because they move the conversation from "read this file" to "how can an agent complete a task on this site?" That is promising, but it is still too early to sell as a standard website requirement.

For now, the safest approach is to build the layer that is useful regardless of which standard wins: clear pages, accurate structured data, crawlable public content, honest machine endpoints, and maintenance.

Layer 5: Ownership After Launch

This is the part most audits miss.

A report can find gaps. A tool can generate a score. A file can list pages.

None of those keep the website correct after the next product launch, campaign page, location update, menu change, pricing change, FAQ update, CMS migration, or agency handoff.

Someone has to own the layer.

That owner checks whether structured data still matches the page. They check whether the sitemap is current. They check whether crawler rules still match the business decision. They check whether answer pages still answer the questions customers ask. They check whether machine-readable files are being requested at all.

AI-readable website work is not a one-time visibility hack. It is maintenance.

What We Would Check First

For a first pass, we would not start by asking whether the site has llms.txt.

We would check:

  • the homepage and key service pages for structured data;
  • robots.txt, sitemap paths, and obvious crawl blocks;
  • FAQ, help, product, booking, event, or service pages for answer-first content;
  • machine-readable support files such as llms.txt, .well-known files, or agent manifests, if present;
  • server-log evidence of useful bot or agent traffic;
  • whether anyone is assigned to keep the layer updated.

That gives a practical answer:

  • what already works;
  • what is missing;
  • what is speculative and should not be overbuilt yet;
  • what needs an owner.

The Difference Between A Report And A Working Layer

An AEO or SEO audit can be useful. It can show what is missing.

The harder question is what happens after the audit.

Who writes the structured data correctly?

Who fixes the pages that answer common buyer questions badly?

Who checks that robots.txt and sitemap paths still match the site?

Who keeps the machine-readable support file from drifting?

Who checks logs and decides whether the work is being used?

That is where Omni Care fits. We do not treat the file as the product. We use a diagnosis to find the gap, turn it into a build plan, implement the useful layer, and keep it maintained as the site changes.

The goal is not to promise that an AI assistant will cite the site tomorrow.

The goal is to stop leaving the website hard to understand, hard to inspect, and hard to maintain for the systems people increasingly use to make decisions.

If your team has already invested in SEO, content, or AEO, the next useful step is not another shortcut. Ask for a website readiness check. We will look at the public technical layer, separate proven work from speculative files, and show what needs to be built or maintained next. Start with the AI adoption assessment to describe what your site already has before a readiness check.

Sources checked for this article