Skip to content
Chalk Labs / AI SEO
How-to guide

How to Run an AI SEO Audit Yourself

By Rahil Jain, founder of Chalk LabsUpdated How we research

The short answer

To run an AI SEO audit yourself, check five things in order: can AI crawlers reach and read your pages, do you have clear answer pages for buyer questions, do trust signals name real people, are you mentioned elsewhere, and what do assistants say about you today. Score 20 checks from 0 to 2, then fix the lowest group first.

  • An audit is a list of checks, a score and a ranked fix list. Half a day is enough for a small site.
  • Start with access: robots.txt, indexing and whether your key text is in the raw HTML.
  • Google says AI features need only an indexed page eligible to show with a snippet 1, and no special schema 8.
  • Test what assistants say about you with a fixed list of prompts, and save the results as your baseline.
  • Fix the lowest-scoring group first. A page nobody can read cannot be fixed by better writing.

What does an AI SEO audit actually check?

An AI SEO audit checks whether AI assistants and AI search features can find, read, trust and quote your pages. It ends with a score and a short, ordered list of fixes. It is not a long report of every possible issue.

We group the checks into five areas: access, content, trust, mentions and measurement. That grouping is our own, built from what the sources below say about how AI features pick pages. For a wider definition see what an AI SEO audit is. If you want to compare doing it yourself with paying for one, read AI SEO audit vs full execution.

Do it step by step

0 of 8 done0%

  1. Open yoursite.com/robots.txt. Look for rules that name OAI-SearchBot, GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot or Googlebot, and for a blanket 'Disallow: /' under 'User-agent: *'. Note any bot you want blocked on purpose.

    Tip: Allowing search bots while blocking training bots is a valid choice. See the guide to blocking AI training but allowing AI search.

  2. In Search Console, paste five important URLs into URL Inspection. Each should say the page is on Google. Note any 'Discovered - currently not indexed' or 'Crawled - currently not indexed' status.

  3. For three key pages, fetch the HTML with curl and search for a sentence from the main answer. If it is missing, the text is built by JavaScript and some crawlers may not see it.

  4. List ten questions buyers ask. For each, find the page that answers it. Mark the check 0 if no page exists, 1 if the answer is buried, 2 if it leads the page.

  5. Open five pages. Look for a named author, a short bio, cited sources, a date, and prices or specs in plain text. Note any structured data that does not match what is visible.

  6. Search your brand and category on review sites, directories and news. Record every listing and any wrong detail such as old prices, closed locations or wrong services.

  7. Run 20 to 30 fixed prompts in two or three assistants. Record whether you were named, linked and described correctly. Save the date and the assistant version if shown.

    Tip: Run each prompt in a fresh chat so earlier answers do not shape later ones.

  8. Fill the scorecard, total it, and rank the findings by how much they block everything else. Access problems come first. Share the report with an owner for each fix.

Your ticks are saved in this browser only.

Copy-ready templates

Three terminal checks (robots, bot status, raw HTML)

bash
# 1. See which AI and search bots your robots.txt mentions
curl -s https://example.com/robots.txt | grep -iE -A3 "user-agent: *(oai-searchbot|gptbot|chatgpt-user|claudebot|perplexitybot|googlebot|\*)"

# 2. See the status code your server returns to a bot-like user agent
curl -s -o /dev/null -w "%{http_code}\n" -A "OAI-SearchBot/1.0" https://example.com/

# 3. Is a key sentence in the raw HTML? (0 means it is built by JavaScript)
curl -s https://example.com/your-page | grep -c "a sentence from your main answer"

Replace example.com. Check 2 only tests rules based on the user-agent text. A firewall that checks IP ranges can still block the real bot, so confirm with your host or server logs.

20-check scorecard (score each 0, 1 or 2)

markdown
ACCESS
1. robots.txt does not block the search bots you want
2. Key pages are indexed in Search Console
3. Key text appears in the raw HTML
4. Pages load fast and return 200

CONTENT
5. Each top buyer question has its own page
6. The answer leads each page
7. Prices, dates and names are in plain text
8. Pages add something original (data, process, examples)

TRUST
9. A named author on key pages
10. Author bio shows real experience
11. Claims cite sources
12. Structured data, if used, matches visible text

MENTIONS
13. Listed on the review sites that matter
14. Name, services and prices match across sites
15. Mentioned in third-party articles or lists
16. Social profiles are current and linked

MEASUREMENT
17. A fixed prompt list exists
18. Baseline mention, citation and accuracy rates saved
19. AI referral visits tracked in analytics
20. A monthly review date is set

TOTAL: ___ / 40   (0-15 fix basics, 16-29 close gaps, 30-40 maintain)

Score the lowest area first when you plan fixes. A 0 in Access outweighs a 2 in Measurement.

What do you need, and how long will it take?

You need about four hours for a site under 100 pages. That is our estimate, not a measured figure. You need access to your robots.txt, Google Search Console and your site's page source, plus a free account with the assistants you want to test.

  • Google Search Console, to see which pages are indexed and which queries bring clicks.
  • A browser with “View page source” and a terminal with curl, to see what a crawler sees.
  • The free AI Crawler Checker and AI Readiness Score for a first pass.
  • A spreadsheet for the scorecard and 20 to 30 test prompts.

Can AI crawlers reach and read your pages?

Start here, because nothing else matters if the page cannot be fetched or understood. Google says a page needs to be indexed and eligible to show in Search with a snippet to appear as a supporting link in AI Overviews or AI Mode, with no additional technical requirements 1.

For ChatGPT, OpenAI says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers. GPTBot is the separate crawler for training, and blocking it signals content should not be used for training 2. A robots.txt that blocks all bots by mistake is a common find. OpenAI says robots.txt changes can take about 24 hours to take effect in its search systems 2.

Then test rendering. A study by Vercel published in December 2024 found that none of the major AI crawlers it tested rendered JavaScript, while Google's Gemini and Apple's crawler did 3. That is old data, so re-check rather than assume. Google itself notes that server-side or pre-rendering is still a great idea because not all bots can run JavaScript 4. If your key text only appears after scripts run, fix that first. Our guide to making your site readable by ChatGPT shows how.

Do you have pages that answer what buyers ask?

List your ten most valuable buyer questions and check that each has one page that answers it in the first two sentences. Google asks whether content provides original information, research or analysis, and goes beyond the obvious 5. A page that only restates common knowledge gives an assistant no reason to pick you.

Check three things on each page: the answer comes first, the page covers one idea, and key facts such as prices, dates and names are in plain text. The Answer Block Checker tests the first part in seconds. For the writing method, see how to write answer-first intros.

Do your pages show who is behind them?

Pages should name a real author, show how they know the topic and cite their sources. Google's self-check asks whether content shows clear sourcing, evidence of expertise and author or site background, and whether the content shows expertise and sourcing 5.

If you use structured data (code that labels page facts for machines), keep it honest. Google says markup must be a true representation of the page and must not cover content readers cannot see 6. It also says structured data isn't required for generative AI search and there is no special schema.org markup to add 8. So the audit question is not whether you have a lot of schema. It is whether what you have is accurate. Our author page guide covers the trust side.

Are you mentioned elsewhere, and what do assistants say about you?

Check your presence on the sites buyers and assistants use: review sites, directories, industry lists, news and your social profiles. Note whether your name, services and prices match your own site. Gaps and contradictions are the findings.

Then run your baseline. Write 20 to 30 prompts across problems, category questions, comparisons and brand questions, and run them in two or three assistants. Keep the list fixed for a month so later runs compare. Semrush advises the same for trend lines 7, and our prompt list guide shows how to build one. Log whether you were named, cited and described correctly, as in the tracking spreadsheet.

Which popular checks can you skip?

For Google Search, Google's AI optimization guide says you do not need to chunk content, create llms.txt or other AI text files, or seek inauthentic mentions 8. Other assistants publish no such rule, so treat this as a priority call, not proof. SE Ranking looked at nearly 300,000 domains in October 2026, found 10.13% had an llms.txt file, and found no link between having one and AI citation frequency 9.

That does not mean the file is harmful. It means it should not score points in your audit. Spend the hour on indexing, rendering and answer pages instead.

How do you score the audit and write it up?

Score each of the 20 checks as 0 (missing), 1 (partly there) or 2 (done well), for a total out of 40. Our bands are 0 to 15 fix the basics, 16 to 29 close the gaps, 30 to 40 maintain and measure. These are our own rules of thumb, not an industry benchmark.

Then write a one-page report: the score by area, the five worst findings with a fix each, an owner and a date. Put the lowest-scoring area first. A reader should know what to do on Monday.

Report outline
SectionWhat goes in itLength
Score summaryTotal out of 40 and the score for each of the five areas3 lines
Top five findingsWhat is wrong, the evidence, the fix, who owns it5 short rows
AI answer baselineMention, citation and accuracy rates from your fixed prompts1 small table
Next 30 daysFixes in order, with dates6 to 8 bullets

Frequently asked questions

Can I really audit my own AI SEO?

Yes, for a small site. Checking crawl access, rendering, answer pages, trust signals and a baseline of AI answers needs no paid tool. An outside reviewer helps when the site is large or you cannot read your own server logs, but the first pass is yours.

Do I need llms.txt or special schema to pass the audit?

No. Google says structured data isn't required and there is no special schema to add 8, and AI features need only an indexed page eligible to show with a snippet 1. SE Ranking found no link between llms.txt and AI citation across nearly 300,000 domains 9. Score accuracy and access instead.

How often should I repeat the audit?

Run the full 20 checks every quarter and the prompt baseline monthly. Re-check access straight after any site rebuild or hosting change, since those are when a bot gets blocked by accident. OpenAI says robots.txt changes can take about 24 hours to apply 2.

What is a good score?

Our own bands are 30 to 40 out of 40 for strong, 16 to 29 for gaps to close and under 16 for basics to fix. These are rules of thumb, not a benchmark from any study. What matters is the trend and the worst area.

Which free tools should I use?

Use Search Console for indexing, curl or View Source for raw HTML, and the free AI Crawler Checker and AI Readiness Score from this library for a fast first pass. Keep your prompt results in a simple spreadsheet.

About the author

Rahil Jain

Rahil Jain is an AI SEO (answer engine and generative engine optimisation) consultant and co-founder of Chalk Labs.

LinkedIn · Method and credentials

Sources

  1. AI features and your website · Google Search CentralOpened October 20, 2026. Supports: A page must be indexed and eligible to show with a snippet to appear in AI Overviews and AI Mode; no additional technical requirements (updated 2025-12-10).
  2. Overview of OpenAI crawlers · OpenAIOpened October 20, 2026. Supports: OAI-SearchBot controls ChatGPT search appearance; GPTBot relates to training; robots.txt changes can take about 24 hours to apply.
  3. The rise of the AI crawler · VercelOpened October 20, 2026. Supports: December 2024 finding that none of the major AI crawlers tested rendered JavaScript, while Gemini (Googlebot infrastructure) and AppleBot did.
  4. Understand JavaScript SEO basics · Google Search CentralOpened October 20, 2026. Supports: Server-side or pre-rendering is still a great idea because not all bots can run JavaScript (updated 2026-03-04).
  5. Creating helpful, reliable, people-first content · Google Search CentralOpened October 20, 2026. Supports: Self-assessment questions on original information, insight beyond the obvious, sourcing, author background and expert review (updated 2026-10-05).
  6. General structured data guidelines · Google Search CentralOpened October 20, 2026. Supports: Structured data must be a true representation of the page and must not mark up content that is not visible to readers (updated 2026-07-10).
  7. How to Build an AI Visibility Dashboard in Semrush · SemrushOpened October 20, 2026. Supports: Keep a core prompt set stable for at least a month because trend lines depend on a constant prompt set.
  8. Optimizing your website for generative AI features on Google Search · Google Search CentralOpened October 20, 2026. Supports: For Google Search: no need to chunk content, create llms.txt or other AI text files, or seek inauthentic mentions; structured data not required and no special schema.org markup for generative AI search (updated 2026-07-10).
  9. llms.txt study of nearly 300,000 domains · SE RankingOpened October 20, 2026. Supports: 10.13% of nearly 300,000 domains had llms.txt; no correlation with AI citation frequency found (15 October 2026).
How this page was made. Built on Rahil Jain's AI SEO method and drafted with AI research assistance. Every statistic links to the page we read it on (opened October 20, 2026 or nearby). Published October 20, 2026; last updated October 20, 2026. Chalk Labs sells AI SEO services, so we have a commercial interest in this topic. Spotted an error? Tell us and we will fix it.