Skip to content
Chalk Labs / AI SEO
How-to guide

How to Create an llms.txt File

By Rahil Jain, founder of Chalk LabsUpdated How we research

The short answer

To create an llms.txt file, write a Markdown file that starts with an H1 site name, adds a one-sentence summary in a blockquote, then lists your key pages as links under H2 headings with short notes. Upload it to your site root so /llms.txt returns plain text. Treat it as a cheap experiment: Google ignores it, and the studies we read found no link to citations.

  • The format is simple: an H1 name, a short blockquote summary, then H2 sections of links with notes. Only the H1 is required.
  • Host it at /llms.txt and check it returns plain text with a 200 status, not your homepage or a 404 page.
  • Google says Google Search ignores llms.txt, and the studies we read found no link to AI citations. Treat the file as an experiment.
  • It suits documentation-heavy sites best. Most small business sites will get little from it.
  • Check the links every month. A stale file points machines at dead pages and old prices.

What is an llms.txt file?

An llms.txt file is a Markdown text file at the root of a website. It gives AI tools a short summary of the site and a curated list of links. Jeremy Howard proposed it on 3 September 2024 1. An LLM, short for large language model, is the kind of AI behind chat assistants.

The proposal describes the file as a way to give background, guidance and links to detailed pages written for LLMs 1. It is a community proposal, not an official web standard. It also differs from robots.txt, which tells crawlers where they may go. Read what llms.txt is and llms.txt vs robots.txt vs sitemap.xml for the differences.

Do it step by step

0 of 9 done0%

  1. Use the rule from this page. Create the file if you run documentation, an API or a developer product, or if you want a cheap test. Skip it if you expect it to win citations for a local or services business, because the studies we read found no link 35.

    Tip: Write down what result would make you keep it, and when you will judge.

  2. Open your robots.txt and check that you have not blocked the AI bots you want. Then check your CDN and firewall. The file does not override robots.txt restrictions 3. The AI Crawler Checker covers the robots.txt part for free.

  3. Choose the pages a careful researcher should read first: offers, pricing, how it works, proof, policies and contact. Put each under one of four or five headings. Leave out login pages, thin tag pages and duplicate URLs.

    Tip: If you cannot say what question a page answers, leave it out.

  4. Start with a single line beginning # and your name. Add a blockquote that says what you do, for whom and where. Use plain facts. No slogans, and no hidden instructions aimed at AI tools.

  5. Put each group under an H2 heading. Write each entry as a Markdown link, then a colon, then a note on what the page answers 1. Put second-rank pages under an H2 named Optional 1.

    Tip: Keep each note under about 15 words, and make it a fact, not praise.

  6. Between the blockquote and the first H2, add a few plain sentences: currency, service area, support hours and the date you last reviewed the file. The proposal allows detail text of any Markdown type except headings 1.

  7. Save it as UTF-8 plain text named llms.txt, in lowercase. Upload it to your site's root folder so it loads at yourdomain.com/llms.txt 1. Or build it with the llms.txt Generator and paste the result.

    Tip: On WordPress or Wix, check first whether a plugin or the platform already created one 6.

  8. Run the response check from the templates. You want status 200, a text content type and an H1 on the first line. A homepage, a 404 page or robots.txt content means the route is wrong 6.

  9. Run the link checker from the templates. Fix any link that does not return 200. Run it again after any site move or redesign.

    Tip: Put a recurring monthly reminder in your calendar, and note the date in the file.

Your ticks are saved in this browser only.

Copy-ready templates

llms.txt skeleton

markdown
# {Your company name}

> {One or two sentences: what you do, for whom, and where.}

{Optional detail text, no headings: currency, service area, support hours, last reviewed on YYYY-MM-DD.}

## {Group name, such as Services}

- [{Page name}]({full URL}): {what this page answers}
- [{Page name}]({full URL}): {what this page answers}

## {Group name, such as Help}

- [{Page name}]({full URL}): {what this page answers}

## Optional

- [{Page name}]({full URL})

Replace every brace with your own text. Only the H1 is required. Use full URLs, including https, and keep each note factual.

Bash: check what /llms.txt really returns

bash
#!/usr/bin/env bash
# Usage: bash llms-check.sh https://yourdomain.com
SITE="${1:?Usage: bash llms-check.sh https://example.com}"
URL="${SITE%/}/llms.txt"
HDR=$(mktemp); BODY=$(mktemp)
code=$(curl -s -L -m 20 -D "$HDR" -o "$BODY" -w '%{http_code}' "$URL")
ctype=$(grep -i '^content-type:' "$HDR" | tail -1 | cut -d' ' -f2- | tr -d '\r')
echo "URL:          $URL"
echo "Status:       $code"
echo "Content-Type: ${ctype:-(none)}"
echo "Bytes:        $(wc -c < "$BODY" | tr -d ' ')"
echo "First line:   $(head -1 "$BODY" | cut -c1-100)"
if grep -q -i -E '<(!doctype|html)' "$BODY"; then echo "WARNING: this looks like an HTML page, not a text file."; fi
if grep -q -i -E '^user-agent:' "$BODY"; then echo "WARNING: this looks like robots.txt content."; fi
case "$(head -1 "$BODY")" in "# "*) ;; *) echo "WARNING: first line is not an H1 (# Name).";; esac
rm -f "$HDR" "$BODY"

Good output is Status 200, a text/plain or text/markdown content type, and an H1 as the first line. We ran this against two real llms.txt files and a site with none, and it flagged the HTML 404 page correctly. It follows redirects, so a redirect to your homepage shows as HTML.

Bash: check that every link in the file works

bash
#!/usr/bin/env bash
# Usage: bash linkcheck.sh llms.txt
FILE="${1:-llms.txt}"
grep -o -E '\]\(https?://[^) ]+\)' "$FILE" | sed -E 's/^\]\(//; s/\)$//' | sort -u | while read -r url; do
  code=$(curl -s -o /dev/null -L -m 20 -w '%{http_code}' "$url")
  printf '%s  %s\n' "$code" "$url"
done | sort

Prints one line per unique link with its final status code. Anything other than 200 needs a look. A redirect that ends at a 200 page passes here, so also read the final addresses by hand after a site move.

HTML: optional link tags from the proposal

html
<!-- In the <head> of a page that has a Markdown copy at /pricing.md -->
<link rel="alternate" type="text/markdown" href="https://yourdomain.com/pricing.md">
<link rel="describedby" href="https://yourdomain.com/llms.txt">

The proposal on llmstxt.org describes these link relations for pointing to a page's Markdown copy and to the covering llms.txt. We found no evidence that any AI service reads them yet. Skip this unless you already publish Markdown pages.

Should you create one at all?

Create one only if you accept that the benefit is unproven. Google says Google Search ignores llms.txt files, so they neither help nor harm your visibility there. Other services may use them 2.

The evidence so far is thin. SE Ranking checked nearly 300,000 domains and found no correlation between having the file and AI citations 3. It did not name the AI engines, queries or dates, so treat that as a warning, not proof. Semrush tested one site from March 2025 and also found no correlation 5.

Use looks low too. Ahrefs found that 97% of the roughly 38,000 valid files in its sample got zero requests in May 2026, from bots or people 4. The Ahrefs author does not recommend one yet, but calls it cheap with little risk 4.

The AI companies do not say they use it for sourcing either. OpenAI's and Perplexity's crawler pages mention llms.txt only to point readers to their own documentation index 910. Neither page says its bots use the file when choosing sources.

The proposal says the file is most used for software documentation 1. That gives a simple rule. Create one if you run documentation, an API or a product that developers explore with coding assistants. Create one as a low-cost test if you can spare an hour. Skip it if you hoped it would win citations for a local or services business.

What do you need, and how long does it take?

A first file takes about one hour. Reviewing it takes about 15 minutes a month. Both times are our estimates, not measured figures. You need the ability to upload a file to your site's root folder, or a plugin that does it for you.

  • A plain text editor. Word processors can add hidden formatting.
  • A list of the 10 to 30 pages you want machines to read first.
  • Terminal access to run the two checks in the templates, or a link checker you trust.
  • Optional: the free llms.txt Generator, which builds the file from a short form.

What goes inside the file, and in what order?

The proposal sets a fixed order, and only the first item is required 1. Everything else is optional. A file with only an H1 is valid, but it tells a reader nothing.

Each link entry needs the Markdown link. A colon and a note after it are optional 1. Write the note anyway, because it is the only place you tell a machine what the page answers.

Aim for the full shape. Common Crawl analysed 584,107 non-empty files from its July 2026 crawl and found that about half (49.90%) had the complete shape the proposal asks for, with an H1, a summary and link sections 6.

The llms.txt layout, in order
OrderElementRequired?What to write
1H1 with the site or project nameYes. It is the only required section 1.Your trading name, exactly as customers know it.
2Blockquote with a short summaryNoOne or two sentences: what you do, for whom, and where.
3Detail text with no headingsNoGround rules such as currency, service area, support hours and a review date.
4H2 sections of linksNoLines like - [Page name](URL): what this page answers.
5An H2 named OptionalNoSecondary links an agent can skip when it needs shorter context 1.

Which pages belong in the file?

List the pages you would want a careful research assistant to read first. That usually means 10 to 30 pages: what you offer, pricing, how it works, proof, policies and contact details. This size is our judgment, not a rule from the proposal.

Real files vary widely. Cloudflare's developer file is an index of about 112 entries, each linking to a product's own llms.txt 7. Anthropic's developer file runs to roughly 600 to 700 links, nearly all to Markdown pages 8. A smaller site should not copy that scale. A short list of pages that answer buyer questions beats a dump of every URL.

Write each note as the question the page answers, or the fact it holds. A note that says Pricing tells a reader nothing. A note that says monthly price of each plan, billing terms and what is included tells them whether to open the link.

Worked example: a made-up invoicing software company
  • # Kestrel Invoicing
  • > Kestrel Invoicing is invoicing software for freelancers and small studios. It sends invoices, tracks payments and chases late payers.
  • Prices are in US dollars. Support is by email on weekdays. This file was last reviewed on 20 October 2026.
  • ## Product
  • - [How Kestrel works](https://kestrel.example/how-it-works): the four steps from first invoice to payment
  • - [Pricing](https://kestrel.example/pricing): monthly price of each plan, billing terms and what is included
  • ## Help
  • - [Set up payment reminders](https://kestrel.example/help/reminders): step-by-step guide with screenshots
  • - [Import invoices from a spreadsheet](https://kestrel.example/help/import): the CSV format and common errors
  • ## Company
  • - [About](https://kestrel.example/about): who runs the company and where it is based
  • - [Security and data handling](https://kestrel.example/security): where data is stored and how long it is kept
  • ## Optional
  • - [Release notes](https://kestrel.example/releases)
  • - [Press kit](https://kestrel.example/press)

Where do you host it, and how do you stop it breaking?

Put it at yourdomain.com/llms.txt. The proposal also allows a file at a subpath such as /docs/llms.txt, which covers URLs under that path 1. The address must return a 200 status and plain text, not your homepage.

That last check matters, because many sites get it wrong. In Common Crawl's July 2026 analysis, 1,287,207 requests for llms.txt and llms-full.txt returned HTTP 200. Of those, 40.36% were HTML pages, which the post attributes to catch-all routes in single-page apps. Another 10.61% were robots.txt served at the llms.txt address 6. Both look fine at a glance and fail for a machine.

Check whether a plugin already makes one. Common Crawl names SEO plugins and website builders among the generators of these files, with median sizes from 3 links (Wix) to 138 (All in One SEO) 6. A generated file may list hundreds of low-value URLs. Open it before you add another.

What you see at /llms.txt, and the fix
What you seeLikely causeFix
Your homepage appearsA catch-all route returns HTML with a 200 status 6.Add a real file, or a route that returns plain text.
A 404 page appearsThe file is not in the root folder, or a cache is stale. A 404 was the most common result in Common Crawl's sample, at 68.4% of requests for /llms.txt 6.Upload it to the root and clear your CDN cache.
The text of robots.txt appearsA rewrite rule sends the address to the wrong file 6.Fix the rule.
The file is blankCommon Crawl found 14,191 empty responses among 598,298 text responses 6.Save the file again and re-upload.
Links in the file return errorsPages moved or were deleted.Run the link checker and update the file.

Do the linked pages need Markdown versions, and what about llms-full.txt?

Markdown versions are optional, but they are what the proposal pictures. It suggests serving a clean Markdown copy of a page at the same address with .md added, and using link relations to point to it 1. Anthropic's file links almost entirely to .md pages 8. For a small site, linking to ordinary pages is a sensible start. Make sure the facts sit in visible text.

A related file, llms-full.txt, holds the content itself in one long file. The proposal page we read does not define it. Anthropic's file points to its llms-full.txt for more comprehensive documentation 8. Few real ones exist: Common Crawl found that only 7.01% of llms-full.txt 200 responses carried a text body, against 52.5% for llms.txt 6. Skip it unless you publish extensive documentation.

How do you keep it from going stale?

Review it every month and whenever pages move. A stale file points machines at dead pages and old prices. Run the link checker below, update the review date in the detail text, and remove anything you no longer want quoted.

Write only what you would print on your website. The Ahrefs author notes one risk: a tidy list of your best pages can make scraping easier for competitors 4. If a page is sensitive, leave it out. Remember that the file controls nothing. It is a reading guide, not an access rule 3.

How do you check that it works?

Check three things: the response, the links and the logs. The response should be status 200 with a text content type, and the first line should be your H1. Every link should return 200. Your logs will probably show little.

Expect quiet logs. In Semrush's test, server logs from mid-August to late October 2025 showed no visits to the file from GPTBot, PerplexityBot or ClaudeBot, and only a few from Googlebot and Bingbot 5. That was one site. If you do see visits, verify the addresses as shown in how to monitor AI bot activity in server logs.

Do not judge success by citations alone. Many things change citations at once, so one file cannot be isolated. For a fairer test, read how to test whether your changes worked.

Frequently asked questions

Does ChatGPT read llms.txt?

We found no OpenAI statement that it does. OpenAI's crawler page mentions llms.txt only to point to its own documentation index 9. In Semrush's test on one site, server logs from mid-August to late October 2025 showed no visits to the file from GPTBot, PerplexityBot or ClaudeBot 5. Treat any benefit as unproven, and judge it by your own logs and results.

Does Google use llms.txt?

No. Google says Google Search ignores llms.txt files, so they neither help nor harm visibility or rankings there 2. The guide still says other services may use them. If you create one, do it for those services, not for Google.

Will an llms.txt file improve my AI citations?

The studies we read found no link. SE Ranking, looking at nearly 300,000 domains, found no correlation between having the file and AI citations, though it did not disclose its sampling method 3. Ahrefs found most files get no requests 4. Expect no lift.

Where should the file go?

Put it at your site's root, so it loads at yourdomain.com/llms.txt. The proposal also allows a file at a subpath, which covers URLs under that path 1. A separate subdomain has its own root, so it needs its own file if you want one there.

How long should an llms.txt file be?

We found no official limit. Real files range widely. Cloudflare's has about 112 entries 7, Anthropic's roughly 600 to 700 links 8, and some generated files only 3 6. For a small business, 10 to 30 well-described links is our suggestion, not a rule.

Is llms.txt the same as robots.txt?

No. Robots.txt is the standard way to tell crawlers where they may go. An llms.txt file only suggests what to read. SE Ranking notes that adding one will not override restrictions in robots.txt 3. See llms.txt vs robots.txt vs sitemap.xml.

About the author

Rahil Jain

Rahil Jain is an AI SEO (answer engine and generative engine optimisation) consultant and co-founder of Chalk Labs.

LinkedIn · Method and credentials

Sources

  1. The /llms.txt file · llmstxt.org (Jeremy Howard)Opened October 20, 2026. Supports: The proposal published 3 September 2024 (page shows version 2, modified 10 August 2026); the order of elements with the H1 as the only required section; link entry syntax with an optional note; the Optional H2 section; file at the root or any subpath; Markdown copies of pages and the alternate and describedby link relations; most used for software documentation.
  2. Optimizing your website for generative AI features on Google Search · Google Search CentralOpened October 20, 2026. Supports: Google Search ignores llms.txt files, which neither help nor harm visibility or rankings in Google Search; other services may use them; last updated 2026-07-10.
  3. LLMs.txt study: Does LLMs.txt impact your AI visibility and citations? · SE RankingOpened October 20, 2026. Supports: Published 15 October 2026: nearly 300,000 domains, 10.13% with an llms.txt file, no correlation with AI citations; engines, queries and period not disclosed; an llms.txt file will not override robots.txt restrictions; the authors call it an optional experiment.
  4. What Is llms.txt? And Do You Need One? · AhrefsOpened October 20, 2026. Supports: Page updated 15 June 2026: of about 38,000 valid llms.txt files in a sample of 137,000 domains using Ahrefs Web Analytics, 97% received zero requests in May 2026; the author does not recommend creating one yet but calls it cheap with little risk; names scraping by competitors as a risk.
  5. What is llms.txt? (Semrush) · SemrushOpened October 20, 2026. Supports: Article dated 28 September 2026: a March 2025 test on Search Engine Land found no correlation with AI results; server logs from mid-August to late October 2025 showed no visits to the file from Google-Extended, GPTBot, PerplexityBot or ClaudeBot and only a few from Googlebot and Bingbot; one site only.
  6. A content analysis of llms.txt files from the July 2026 crawl archive · Common CrawlOpened October 20, 2026. Supports: 584,107 non-empty files analysed; 49.90% have the full shape the spec asks for; of 1,287,207 HTTP 200 responses 40.36% were text/html and 10.61% were robots.txt content; 68.4% of /llms.txt requests returned 404; 14,191 empty responses among 598,298 text responses; text-body rates of 52.5% for llms.txt and 7.01% for llms-full.txt; per-generator median link counts; the sample is not representative of the web.
  7. Cloudflare Developer Documentation (llms.txt) · CloudflareOpened October 20, 2026. Supports: A real llms.txt: H1, one intro sentence, a blockquote, nine H2 groups and about 112 entries in the form [name](URL): description, each linking to a product-level llms.txt.
  8. Anthropic Developer Documentation (llms.txt) · AnthropicOpened October 20, 2026. Supports: A real llms.txt: roughly 600 to 700 links, nearly all to .md pages, with optional descriptions and a pointer to llms-full.txt for more comprehensive documentation.
  9. Overview of OpenAI Crawlers · OpenAIOpened October 20, 2026. Supports: The page mentions llms.txt only in its opening note, which points to OpenAI's own documentation index; it does not say its bots use llms.txt files from other sites.
  10. Perplexity Crawlers · PerplexityOpened October 20, 2026. Supports: The page mentions llms.txt only in its opening line, which points to Perplexity's own documentation index; it does not say its bots use llms.txt files from other sites.
How this page was made. Built on Rahil Jain's AI SEO method and drafted with AI research assistance. Every statistic links to the page we read it on (opened October 20, 2026 or nearby). Published October 20, 2026; last updated October 20, 2026. Chalk Labs sells AI SEO services, so we have a commercial interest in this topic. Spotted an error? Tell us and we will fix it.