Skip to content
Chalk Labs / AI SEO
How-to guide

How to Decide Where to Host Your AI SEO Content

By Rahil Jain, founder of Chalk LabsUpdated How we research

The short answer

Put your AI SEO content in a subfolder of your main site, such as example.com/guides/, unless a technical or legal reason forces a subdomain. Google staff have said in the past that either works, so the choice is a weak ranking lever. A subfolder is simply easier: one set of settings, one set of reports, and no new robots.txt file to forget 12.

  • Google staff have said, in older remarks reported by Ahrefs, that either structure is fine 1.
  • A subdomain is a separate host, so it needs its own robots.txt file and its own Search Console setup 23.
  • Moving later is possible but slow: Google says to keep redirects for at least a year 5.
  • Choose by who controls the platform and how tied the content is to your main site, not by a ranking myth.

Which should you choose, a subfolder or a subdomain?

Choose the subfolder unless you cannot. A subfolder is a path on your main host, like example.com/guides/. A subdomain is a different host, like guides.example.com. Search engines can treat both as part of your site, but a subfolder keeps your content, links and settings in one place.

Ahrefs reviewed what Google staff have said. Ahrefs reports that Google staff called the two roughly equivalent (Cutts, 2012) and fine to use (Mueller, 2017), and advises thinking twice before moving only for SEO 1. These are older statements reported second-hand, not a current Google documentation page, so treat them as guidance, not a rule.

Nobody has published a controlled test that proves a subfolder lifts AI citations. We found no reliable data on that question for ChatGPT, Perplexity or Gemini. The advice here rests on practical costs, not on a claimed ranking boost.

Do it step by step

0 of 8 done0%

  1. Write down every host your site uses: www, blog, help, docs, shop. Note which one holds your best-performing content and which platform runs each.

  2. Ask your developer or vendor whether your main host can serve a new path, such as /guides/. Get the answer in writing, including cost.

    Tip: A reverse proxy or routing rule can often serve a separate app at a path.

  3. Choose the subfolder if the platform allows it. Choose a subdomain only for a stated reason: separate platform, separate team, server location or separate product.

  4. Create or confirm a robots.txt file on each host you use. Allow the search crawlers you want and test with the AI crawler checker.

    Tip: Remember that a change can take about 24 hours to reach OpenAI's systems 7.

  5. Use a Domain property so subdomains and subfolders are all covered 3. Make sure analytics tracks every host in one view.

  6. Link from your home page or navigation to the new section, and from each guide to the related product or service page.

  7. List every new URL in a sitemap on the same host as the pages, with a real lastmod date, and submit it in Search Console.

  8. Map each old URL to its new one with a 301. Re-check coverage, clicks and citations weekly for eight weeks, and keep the redirects for a year or more 5.

Your ticks are saved in this browser only.

Copy-ready templates

robots.txt for each host (allow search, one file per host)

text
# Place a copy at https://example.com/robots.txt
# and at https://guides.example.com/robots.txt if you use a subdomain
User-agent: OAI-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Googlebot
Allow: /

Sitemap: https://example.com/sitemap.xml

OAI-SearchBot and PerplexityBot are search crawlers; training crawlers such as GPTBot are separate settings . Edit the sitemap line on each host. Decide your training-bot policy separately. This is an allow-list starter, not a full policy; add any Disallow rules you need.

Nginx: redirect a subdomain to a subfolder

text
server {
  listen 443 ssl;
  server_name guides.example.com;
  # ssl_certificate lines go here
  return 301 https://example.com/guides$request_uri;
}

Test one URL first, for example guides.example.com/what-is-aeo should land on example.com/guides/what-is-aeo. Keep this rule for a year or more .

Decision log entry

markdown
## Hosting decision for AI SEO content
- Date: 
- Chosen: subfolder / subdomain
- Reason: 
- Owner of platform: 
- robots.txt checked on every host: yes / no
- Search Console Domain property: yes / no
- Review date (6 months):

Keep it in your wiki so the reason is not lost.

What do you need before you decide?

You need three facts: who controls your main website platform, where the new content will be built, and whether you already have a live blog or help centre on a subdomain. Allow one hour to gather them and a short call with your developer.

If you have no developer, ask your platform vendor one question: can we serve a /guides/ path from the same domain? Many hosted tools support it, some only via a paid plan, and some force a subdomain.

Does a subfolder share authority with the main site better?

There is no firm evidence that it does. The idea that subfolders inherit more strength comes from older studies, and Ahrefs notes that sites linked well to the main domain tend to behave as part of it 1.

Real migrations are messy. Ahrefs describes the mention.com move from a subdomain to a subfolder in 2016: the author saw no real difference, and traffic grew only after new content was added. The GitHub blog moved twice, and other changes made the result impossible to credit to the URL 1.

What matters more is internal linking. Link your guides from your main pages and your main pages from your guides, so a visitor and a crawler can move between them. Our guide on internal linking for AI search shows how.

What extra work does a subdomain create?

A subdomain doubles several jobs. Google states that a robots.txt file applies only to the host, protocol and port where it is hosted, and that a file on a subdomain is valid only for that subdomain 2. If guides.example.com has no file, or a blocking one, your main file does not help. That matters for AI crawlers too, so check each host with the AI crawler checker.

Reporting splits as well. In Search Console, a Domain property covers all subdomains, while a URL-prefix property excludes other subdomains and needs one property per subdomain 3. Analytics, sitemaps, schema and canonical tags also need separate care.

OpenAI documents that each of its bot settings is independent, and that a robots.txt change can take about 24 hours to take effect 7. So a mistake on a subdomain can keep your pages out of ChatGPT search for at least that long after you fix it.

When is a subdomain the right answer?

Use a subdomain when the content runs on a different platform you cannot proxy, when a different team must own the code and release schedule, when you need a different server location, or when content serves a clearly separate product. Google's own table lists easy setup, different server locations and easy separation of sites as strengths of subdomains, with unclear geotargeting as the main weakness 4.

There is one odd AI-era twist. Google's preferred sources tool, which can also surface sites in AI Mode and AI Overviews, accepts only domain-level and subdomain-level sites. A subdirectory such as /blog is not eligible 6. The page does not say how much this affects results, so it is not a reason to move a whole blog. It is a reason to be aware of the rule if a separate news or research site is part of your plan.

Subfolder or subdomain: a side-by-side check
QuestionSubfolder (example.com/guides/)Subdomain (guides.example.com)
Setup effortLow if your platform allows itLow, works with any platform
robots.txtOne file for the whole site 2One file per subdomain 2
Search ConsoleCovered by Domain or main propertyNeeds its own URL-prefix property, or a Domain property 3
Different tech stackNeeds a proxy or routing ruleEasy to separate 4
Ranking evidenceNo proven advantage 1No proven disadvantage 1
Preferred sources toolNot eligible as a subdirectory 6Eligible 6
Cost of changing laterRedirects needed, keep a year or more 5Same

What is a simple decision rule?

Ask four questions in order. Can your main platform serve the path? If yes, use the subfolder. Does a separate team or platform own the content? If yes, the subdomain is acceptable. Do you already have a working blog on a subdomain with traffic? If yes, leave it and improve it. Do you plan to publish many pages? If yes, choose now, because Google says a move with URL changes can take a few weeks or more to settle even for a medium site 5.

Write the decision on one page with the date and the reason. A future colleague should be able to see why you chose it.

How do you move content if you chose wrong?

Use permanent server-side redirects, such as 301 or 308, from every old URL to its exact new URL. Google says to keep them as long as possible, generally at least a year, and that a medium-sized site can take a few weeks or more to show the new URLs 5. It also notes that the Change of Address tool applies when moving from one domain or subdomain to another 5.

AI assistants sometimes send people to old or invented URLs, so redirects also protect you there. See how to capture hallucinated URLs with 301 redirects. Update your sitemap, canonical tags and internal links on the day of the move.

What mistakes do teams make?

The usual errors are launching on a subdomain and forgetting its robots.txt, redirecting every old URL to the home page, leaving two live copies of the same page, and changing structure and content in the same month so you cannot tell what worked. Google also reminds site owners that duplicate content should be reduced for good results in its AI features 8.

The fix for all four is a short checklist and one change at a time. The steps below give it in order.

Frequently asked questions

Does Google treat subdomains as separate sites?

Sometimes, and it depends on how the site is built and linked. Ahrefs reports Google staff saying either structure is fine, and that integrated subdomains tend to act as part of the main site 1. Technically they are separate hosts with separate robots.txt files 2.

Will a subfolder help me get cited by ChatGPT?

We found no reliable data showing that it does. What we can say is practical: one robots.txt file, one set of settings and easier internal linking reduce the chance of a mistake that hides your pages from crawlers 27.

Can I use a different language on a subfolder?

Yes. Google lists subdirectories as an easy, low-maintenance way to serve locale-specific URLs, with the weakness that separation of sites is harder 4. Add hreflang tags so each version links to the others.

Do I need a new Search Console property for a subdomain?

If you use a Domain property, no. It covers all subdomains and subfolders. A URL-prefix property does not include other subdomains, so each one needs its own 3.

About the author

Rahil Jain

Rahil Jain is an AI SEO (answer engine and generative engine optimisation) consultant and co-founder of Chalk Labs.

LinkedIn · Method and credentials

Sources

  1. Subdomain vs. Subfolder: Which Is Better for SEO? · AhrefsOpened October 20, 2026. Supports: Ahrefs reports Matt Cutts (2012) and John Mueller (2017) saying either structure is fine; conclusion that structure is not a meaningful factor; mention.com and GitHub migration cases. Published 9 March 2021; second-hand reporting of older Google statements.
  2. How Google interprets the robots.txt specification · Google Search CentralOpened October 20, 2026. Supports: Rules apply only to the host, protocol and port where the file sits; a robots.txt on a subdomain is only valid for that subdomain. Last updated 2026-08-31.
  3. Add a website property to Search Console · Google Search Console HelpOpened October 20, 2026. Supports: Domain property covers all subdomains and protocols; URL-prefix property excludes other subdomains and needs one property each.
  4. Managing multi-regional and multilingual sites · Google Search CentralOpened October 20, 2026. Supports: Google's table of pros and cons for ccTLDs, subdomains and subdirectories. Last updated 2025-12-10.
  5. Site move with URL changes · Google Search CentralOpened October 20, 2026. Supports: Use 301 or 308 server-side redirects; keep them generally at least a year; a medium site can take a few weeks or more; Change of Address tool applies to moves between domains or subdomains.
  6. Preferred sources · Google Search CentralOpened October 20, 2026. Supports: Only domain-level and subdomain-level sites are eligible for the source preferences tool; a subdirectory is not; preferred sources can also appear in AI Mode and AI Overviews. Last updated 2026-09-18.
  7. Overview of OpenAI crawlers · OpenAIOpened October 20, 2026. Supports: OAI-SearchBot surfaces sites in ChatGPT search; GPTBot is for training; each setting is independent; robots.txt changes can take about 24 hours to take effect.
  8. Optimizing your website for generative AI features on Google Search · Google Search CentralOpened October 20, 2026. Supports: Recommends crawlability, good page experience and reducing duplicate content. Last updated 2026-07-10.
How this page was made. Built on Rahil Jain's AI SEO method and drafted with AI research assistance. Every statistic links to the page we read it on (opened October 20, 2026 or nearby). Published October 20, 2026; last updated October 20, 2026. Chalk Labs sells AI SEO services, so we have a commercial interest in this topic. Spotted an error? Tell us and we will fix it.