What Is llms.txt? A Site Summary for AI Tools

If you have seen llms.txt mentioned in SEO threads and wondered whether your site needs one, the short answer is this: llms.txt is a proposed plain-text Markdown file, placed on your site, that gives AI tools a compact, hand-curated summary of what the site is and where its key pages are. It is a community proposal, not an internet standard, and Google has stated in its own documentation that Google Search does not use it.

This guide explains what the file is, what the proposal says about its structure, how it differs from robots.txt and sitemap.xml, what belongs in it, and how to publish one without creating problems. It also keeps the claims honest: nothing here promises rankings, traffic or inclusion in AI answers. The sample file structures and domains are fictional, and no measurement data is invented.

What Is llms.txt?

A short definition

llms.txt is a small Markdown file, usually served at the root of a site as /llms.txt, written so that both people and language-model-based tools can quickly grasp what a site offers. According to the proposal published on llmstxt.org (Opens in a new tab), the file gives a curated overview, and an AI agent can then follow the links inside it to fetch details on demand. The proposal stresses that the file should stay small enough to fit in a model's context, while the longer material lives behind the links.

The name deliberately echoes robots.txt and sitemap.xml, but the job is different. Think of it as a short briefing note written by the site owner, not a rulebook and not an index of every URL.

Who proposed it, and when

The llmstxt.org page credits Jeremy Howard and dates the proposal to September 3, 2024. At the time of reading, the page described itself as version two and showed a modification date of August 10, 2026. The page also notes that the format has seen most use in software documentation. That context matters: a developer-tools company whose docs change weekly has a different need from a five-page business site, a news publisher or a marketplace.

What llms.txt is not

Much of the confusion comes from expecting the file to do jobs that belong elsewhere. llms.txt is not an access-control file, so it does not allow or block any crawler. It is not an indexing instruction, so it cannot get a page into Google or keep one out. It is not defined as a ranking signal, a licensing statement or a copyright notice. Each of those needs another tool: robots.txt for crawl access, noindex or authentication for keeping pages out of search results, sitemaps and internal links for discovery.

The Proposed Structure of an llms.txt File

Sections, in order

The proposal describes a specific order for the file. The table below summarizes it.

That ordering lets a person read the file as a mini-brief and lets a simple parser pull out the title, the summary and the link lists. The proposal also mentions an optional byte-order mark at the start, but plain UTF-8 text is normally all you need.

A fictional example, described step by step

Imagine a fictional publisher, Example Daily. Its file would read as follows, from top to bottom:

• The first line is a level-one heading with the site name.
• The next line is a one-sentence blockquote saying who publishes the site, in which languages, and which topics it covers.
• A short free-form paragraph follows, covering how often content is updated and where the editorial policy lives.
• A level-two heading such as Guides opens a list of three to five links, each written as a Markdown link followed by a colon and a single factual sentence.
• A final level-two section called Optional holds secondary pages such as an archive, an about page or a contact page.

Use absolute URLs in the links. A tool that reads the file out of context has no page address to resolve relative paths against. Keep the notes factual: a sentence saying what the page contains is more useful than a slogan.

Markdown versions of pages and the Optional section

The proposal goes one step further and recommends that pages with useful information also offer a clean Markdown version at the same URL with .md appended, or index.md for addresses that have no file name. It suggests signaling the Markdown version through an alternate link in the HTML or an HTTP Link header. This step is optional and not worth it for every site. On a documentation site generated from Markdown, it may come almost for free. On a small blog, the maintenance cost can outweigh any benefit. The Optional section is the lighter-weight tool: put the essential links first and park the rest at the bottom, where a constrained tool may skip them.

llms.txt vs robots.txt vs sitemap.xml

Three files, three jobs

These files sit side by side in a site's root, which tempts people to treat them as alternatives. They are not. Google's robots.txt documentation says the file is mainly for managing crawler traffic so that a server is not overwhelmed, and that it is not a mechanism for keeping a web page out of Google; for that, it recommends noindex or password protection. llms.txt takes over neither of those roles.

The llmstxt.org page describes the contrast in its own words: robots.txt states what automated access is acceptable, while llms.txt is consulted on demand when an agent is answering a user. A sitemap, it adds, lists everything indexable and may be too big for a context window, whereas llms.txt is curated. So llms.txt lives next to your sitemap; it does not replace it.

What goes where

If you want to block a bot, writing that into llms.txt does nothing; the rules belong in robots.txt. Leaving a page out of llms.txt does not remove it from a search index; use noindex for that. For a hands-on look at crawl rules, see our robots.txt audit guide. For page-level exclusion, see the noindex tag and X-Robots-Tag guide. For your URL list, the XML sitemap audit and error fixes article covers the checks.

Does Google Use llms.txt?

What Google's documentation says

Google addresses the question directly in its guide to optimizing for generative AI features on Google Search, which showed a last-updated date of July 10, 2026 when we read it. In its myth-busting section, the guide says you do not need to create new machine-readable files, AI text files, markup or Markdown to appear in Google Search, because Google Search does not use them. It adds that it is completely fine to create and maintain an llms.txt file for other services, and that doing so will neither harm nor help visibility or rankings in Google Search, as Google Search ignores them. You can read the original wording in Google's generative AI optimization guide (Opens in a new tab).

The same guide says there is no requirement to break content into tiny pieces for AI, and no special schema.org markup is needed for generative AI search, while structured data is still recommended for rich result eligibility.

The requirement for AI Overviews and AI Mode

A second Google page, the one about AI features and your website, says that to be eligible as a supporting link in AI Overviews or AI Mode, a page must be indexed and eligible to be shown in Google Search with a snippet, and that there are no additional technical requirements. It also says no other special optimizations are necessary. That page was last updated on December 10, 2025. See Google's AI features page (Opens in a new tab) for the exact text.

What this does and does not tell you

Google's statements cover Google Search only. Whether some other AI product reads llms.txt is a question for that product's own documentation, and this article does not claim otherwise. That means the popular line that llms.txt boosts AI visibility is not a verified fact. On the other hand, saying that it is harmless, cheap and possibly useful to some tools is a judgment call, not proof. Set your expectations accordingly.

If you want the wider picture of how content gets used by AI-powered search, our generative engine optimization guide gives an overview. As it explains, the basis of everything is still useful, accessible, well-structured content.

What to Put in llms.txt, and What to Leave Out

What to include

A good file lists the pages you would pick if you had two minutes to introduce your site to a stranger. In most cases that means:

• The site name and who publishes it.
• One or two sentences on scope: topics, languages and audience.
• Stable, public core pages such as about, contact, main category pages and in-depth guides.
• A short factual note beside each link describing the page.
• Context notes where relevant, such as editorial, translation or update policies.

For multilingual sites, separate sections per language keep the file readable. Write the URLs exactly as your canonical addresses so the file never contradicts the site itself.

What to leave out

The file is public text. Anything private, gated or unfinished does not belong in it. Admin paths, staging sites, pages with personal data, and URLs that carry tokens or keys should never be listed; if an address needs to stay hidden, mentioning it in a public file is a mistake in itself. Pages you have set to noindex should not appear either, since listing them sends a signal that contradicts your own directives and adds maintenance work.

Avoid promotional claims such as best, number one or guaranteed. They do not serve the file's purpose. Avoid stuffing dozens or hundreds of links, which goes against the proposal's idea of a small, curated entry point. A small set of well-chosen links works for many sites; the proposal sets no numeric limit on links, so do not treat any specific number as a rule. Larger sites can add sections; the aim is still to introduce the site, not to dump it.

Tone, language and freshness

Write notes in plain language. Say what a page contains instead of praising it. When a page is removed or its address changes, update the file, because a summary full of broken links erodes trust faster than having no summary. The file does not need a date line, but on large sites that change often, adding a quick check to the publishing routine is sensible.

How to Create an llms.txt File, Step by Step

Step 1: choose your pages

Start with an inventory. List the public pages that would help a newcomer understand the site, and mark which ones are the backbone and which are supporting. This is really a content-prioritization exercise, and the same thinking underlies a content marketing strategy for SEO.

Step 2: write the file

Open a text editor, and save the file as UTF-8. For most sites the following sequence is enough:

1. Create a file named llms.txt.
2. Put the site name on the first line as an H1.
3. Add a one-sentence blockquote that describes the site.
4. Optionally add a few lines of free-form notes about scope or usage, without headings.
5. Add H2 sections grouped by topic, each with Markdown links and brief notes.
6. Move secondary links into a final section called Optional.
7. Write every link as a full, canonical URL.

Writing the file by hand is often the healthiest approach for small and mid-sized sites. If you generate it automatically, remember that choosing which pages qualify is still an editorial decision; even if you encode the rule in software, the rule should be public, published and indexable pages only.

Step 3: publish it

According to the proposal, the file can sit at the site root or at any path, and it covers the pages beneath that path; when several files apply, the most specific one is used. For most sites, the root address /llms.txt is the right place. Make sure the server returns it as plain text with UTF-8 encoding and a 200 status, and that it opens without a login.

Step 4: test after publishing

Run through this short checklist:

• Does the address open in a browser and display as plain text?
• Does every link return a 200 response, with no redirect chains?
• Is every listed page public and indexable?
• Does the file contain any hidden URLs, test areas or personal data? It must not.
• Do the language sections point to the right pages?

Also confirm that robots.txt is not blocking the file itself. A blocked file cannot do its job. Having a colleague run the same checks is worthwhile, since site owners are poor at spotting their own blind spots.

Who Benefits From llms.txt, and Who Can Skip It

Documentation sites and complex structures

The proposal's strongest use case is software documentation, where an assistant needs fast access to the current docs of a specific library. Any site with many sections, versions or technical pages can benefit from a tidy introduction, for people as much as for tools. Training sites, large guide libraries and product documentation fit this group.

Small blogs and brochure sites

On a site with five pages, the file adds almost nothing. If the home page is clear, the menu is logical and a sitemap exists, the real work is already done there. For such sites, llms.txt is a small extra, not a growth tactic. Spend your time first on crawlable pages, clear headings, internal links and trustworthy content. Explaining brands and topics to machines is largely the territory of entity SEO.

Marketplaces, news sites and publishers

Sites with thousands of listings, products or articles run into a size problem: they cannot list everything. In such cases the file works better when it points to category pages, policy pages and a guides archive instead of individual listings. A news site can list its main sections and editorial standards so that the file functions like an identity page. The principle stays the same: selected, current, public content.

Common Mistakes

Expectation mistakes

The first mistake is treating the file as a ranking or AI-visibility switch. Google has documented that Search does not use it, and the behavior of other providers should be verified from their own documentation. The second is neglecting core SEO work because the file now exists; it does not compensate for broken links, thin content or crawl problems. The third is believing services that promise traffic gains from an llms.txt file. If someone guarantees an outcome, be cautious.

Technical mistakes

Frequent technical problems include serving the file as HTML or in the wrong character encoding, using relative or broken links, mixing up language sections, listing login-only pages and listing pages that are set to noindex. Another common one is leaving old hostnames in the links after a domain change. One practical defense is to write the origin as a placeholder and substitute the real address when the file is served, but that is a convenience, not a requirement.

Content mistakes

Piling in hundreds of links, turning the notes into keyword lists and calling every page the most complete guide all go against the spirit of the file. Notes should be short and accurate. A sentence you write about a page should be verifiable when someone opens that page; otherwise it harms trust with both human and automated readers.

Maintenance and Realistic Measurement

How often should it be updated?

There is no fixed schedule. Review the file when you publish a new core page, remove a section or change your URL structure. On sites with regular publishing, adding an item such as is llms.txt still accurate to the release checklist turns maintenance into a single step.

Can you measure the effect?

Honestly, only partly. Google documents that Search ignores the file, so waiting for movement in Search Console would make little sense. You may be able to see requests from other tools in server logs, but it is hard to attribute a visit to the file as opposed to any other page. For that reason, measure success by whether the file is accurate, current and simple, not by rankings or traffic. A realistic goal makes for a sounder decision.

Should the file itself be indexed?

llms.txt is not designed as a search result page. Having it appear in Google's index is not a goal. You may consider keeping it out of the index with a server header if that suits your setup, but this is neither required nor a universal recommendation. Whichever you choose, make sure that robots rules or security settings do not leave the file unreachable by accident.

A Worked Example: A Bilingual Marketplace

Akua Adforce's own file

Akua Adforce publishes its own /llms.txt. The file follows the proposed format: a site name at the top, a blockquote describing the marketplace, and then Turkish and English sections of public page links. These include the home page, the about and contact pages, the how-it-works page, backlink packages, guest post packages, footer links, ad spaces, the pricing and commission policy and the SEO guide archive. A note inside the file states that it lists only published, publicly accessible pages, that it does not replace robots.txt or the sitemap, and that it is not a guarantee of search or AI indexing. This is only an illustration of one implementation; it carries no claim of results.

Lessons from the example

Three things stand out. First, the file contains no gated or hidden paths, only public pages. Second, languages are kept in separate sections, which makes it clear which links are for which readers. Third, stating the limits of the file up front prevents readers from forming the wrong expectations. You can apply the same three principles on your own site.

Decision Guide: Should You Write One?

A quick self-check

Before you commit, ask yourself a few questions. Is your site hard to explain to a newcomer? Do you have a stable set of public core pages? Is there someone who will keep the file current? If the answers are yes, a small file is a reasonable addition. If not, it is more productive to improve site structure, page quality and crawl access first.

Cost and benefit

The cost is low: a few hours of editorial work and a routine check. The benefit is uncertain: Google does not use the file for Search, and other tools may behave differently and may change over time. That balance makes llms.txt optional and low-risk rather than necessary. If you are a content publisher, spending most of that time on articles that answer what readers actually search for is a more concrete investment.

If you need help on the content side, you can browse content writing listings on Akua Adforce. The scope and terms of each listing are written on the relevant product and policy pages, and Akua does not promise that an llms.txt file will bring rankings, traffic or AI visibility.

Frequently Asked Questions

What is llms.txt and what is it for?

llms.txt is a Markdown file placed at a site's root that introduces the site to AI tools through a short summary and selected links. It is a community proposal, not a standard. Its aim is to let a tool grasp the site quickly and fetch details through the links. It sets no access rules and does not produce rankings.

Does llms.txt affect Google rankings?

According to Google's published documentation, no. Google Search does not use these files and ignores them, so creating one neither harms nor helps your visibility or rankings. You can still write one for other tools, but do not expect it to lift your Google performance.

Does llms.txt replace robots.txt?

No. robots.txt manages crawler access, while llms.txt is a summary guide. Use robots.txt to allow or block bots. To keep a page out of Google's index, use noindex or password protection.

How do I create an llms.txt file?

Create a text file called llms.txt. Start with an H1 containing the site name, add a short blockquote summary, then list Markdown links with brief notes under H2 headings and place secondary links under an Optional section. Save it as UTF-8, upload it to the site root and test every link.

Which pages should go into llms.txt?

Only public, stable, indexable pages: about, contact, main category pages and in-depth guides. Leave out login-only, test, personal-data and noindex pages. Keeping the number of links small helps the file stay curated.

Does every website need llms.txt?

No. On small and simple sites the added value may be limited. Prioritize crawlability, content quality and internal linking first, and treat the file as a low-cost optional extra.

Conclusion and Next Steps

llms.txt is a short Markdown file in which a site introduces itself, and it remains a proposal rather than a standard. Google has documented that Search does not use it and ignores it, while the behavior of other AI products must be checked individually. If you choose to write one, keep expectations low and keep the file accurate, current and public.

The next steps are short: pick your core public pages, write the file and test its links, and then review your robots.txt and sitemap as well. For a broader view, continue with our guide to content for AI-powered search.