A customer used to find you by typing a phrase into Google and clicking a blue link. Now they ask ChatGPT which supplier to call, and a paragraph answers back. Your name is in that paragraph or it is not.
That shift has business owners asking a fair question: can these tools even read my website? The answer is yes, mostly, and in a narrower way than you would expect. AI systems do not browse. They fetch, and what they can fetch depends on how your site was built.
Almost nothing here is new or exotic. A website AI systems can read is one that is well structured, publicly accessible, and clearly written. Here is what actually happens when an AI tool visits, an honest read on llms.txt, and a 10-point check you can run this afternoon.
Does ChatGPT Actually Read Your Website?
Yes, though “read” is doing a lot of work in that sentence.
A person opens your homepage, waits for it to load, watches the animation finish, and scrolls. An AI system does none of that. It requests the raw HTML file your server sends, takes the text out of it, and moves on. Think of a librarian photocopying a page rather than a visitor touring your office.
Different platforms fetch that page for different reasons. OpenAI documents four separate crawlers, and they do not do the same job:
- OAI-SearchBot powers ChatGPT’s search results. Block it and your site cannot appear there.
- GPTBot collects content for model training. Blocking it has no effect on ChatGPT search.
- ChatGPT-User fetches a page live, at the moment a user asks something.
- OAI-AdsBot checks ad landing pages.
Those are separate switches. Plenty of businesses block GPTBot to keep their content out of training, then wonder why nothing changed in ChatGPT search. Different bot.
Google works differently again. AI Overviews and AI Mode draw on the index Googlebot already builds, so your regular search foundations carry over. Perplexity runs its own crawler. The common thread: all of them need your content public, fetchable, and present in the HTML.
The JavaScript Trap
Here is the finding that surprises most owners. As of 2026, none of the major AI crawlers run JavaScript. Vercel’s analysis of AI crawler traffic found that GPTBot and ClaudeBot download JavaScript files and never execute them.
Why does that matter? Say your service pages or pricing load after the page arrives, through a script. An AI crawler sees an empty shell. Your site looks perfect to you. To ChatGPT, half of it does not exist.
Googlebot renders JavaScript, so a site like this can rank in classic search while staying invisible to AI answers. If your site runs on a modern JavaScript framework, check this first. It is a website development question rather than a marketing one.
What AI Crawlers Actually Look At
Once a crawler has your HTML, it is working with a fairly short list of signals.
- The visible text. The single largest input. If a claim about your business is not written in words on a page, no system can repeat it.
- Heading hierarchy. One H1 stating what the page is, H2s for its main sections, H3s beneath those. Headings are the table of contents a machine reads first.
- Semantic HTML. Real headings, lists, and paragraphs rather than styled <div> soup. The tags tell a parser what each block is.
- Internal links. How a crawler discovers your other pages, and how it learns which topics you cover in depth.
- Metadata. Title tags and descriptions still summarize the page.
- Image alt text. The only description of an image a text-based system receives.
- Structured data. Machine-readable facts about your business, covered below.
- Crawlability. Anything behind a login, a form, a PDF, or a noindex tag is effectively invisible.
That last point catches people out. A price list published as a PDF, or a services overview baked into an image, may as well be blank paper. What a machine cannot parse, it cannot quote.
What Is llms.txt?
llms.txt is a proposed standard. It is a plain text file at your domain root, listing your most important pages in a clean format, so an AI system can read a tidy summary of your site instead of crawling everything.
People compare it to robots.txt. That helps you remember where it lives and misleads you about what it does. robots.txt tells crawlers what they may not visit, and every major crawler obeys it. llms.txt suggests what they should read. Nothing obliges anyone to look.
The Honest Assessment
Here is where marketing advice has run ahead of the evidence.
Ahrefs studied 137,000 domains and found that 97% of valid llms.txt files received zero requests in May 2026. Not zero from AI bots. Zero from anything. Of the files that did get requested, most of the traffic came from SEO audit tools.
Google is blunter. Its guidance on AI features says “you don’t need to create new machine readable files, AI text files, or markup to appear in these features.” Google’s Search team has compared llms.txt to the old keywords meta tag: a file sites dutifully filled in while no search engine read it.
So ignore it? Not quite. The file takes an hour, breaks nothing, and may matter if adoption arrives. Publish one if it is cheap for you. Do not let anyone sell it to you as an AI visibility strategy, and do not expect it to move anything today.
Does Schema Markup Help AI Understand Your Website?
Schema markup is structured data. It is a small block of code stating plainly what a page is about, in a format machines parse without guessing.
Your contact page might say “Verta Marketing, Toronto, open 9 to 5.” A human reads that instantly. Schema turns it into labelled fields, so a machine knows Toronto is a location rather than a client name.
The types that earn their keep for most businesses:
- Organization schema confirms who you are, where you are, and how to reach you. It is the backbone of what search systems call entity clarity.
- Service or Product schema defines what you sell.
- FAQPage schema (Getting retired by Google) labels your questions and answers as questions and answers.
- Author schema attaches a named human to your content, which supports credibility signals.
- LocalBusiness schema matters for anyone with a location or a service area.
Now the caveat. Google says there is “no special schema.org structured data that you need to add” to appear in AI features. Schema is not a shortcut to a citation, and nobody can promise you one.
What it does is remove ambiguity. Consistent facts across your site, your schema, and your Google Business Profile make you easier to identify and harder to confuse with a competitor. That is plumbing, not magic, and plumbing is what fails.
Why Website Structure Matters More Than Ever
A machine understands your business by mapping the shape of your site. A sprawling site with one bloated “Services” page tells it almost nothing.
- One page per service. If you offer six services and they share a page, no system can cite you as an authority on any single one. Six pages, six clear topics.
- Logical hierarchy. Home, then category, then detail. Every page reachable in three clicks.
- Internal links between related pages. Links carry meaning. A service page linking to a relevant case study says these two things belong together.
- Topic clusters. A pillar page covering a subject broadly, supported by deeper pages, is the clearest way to show expertise.
- Readable URLs. /website-maintenance-cost/ explains itself. /page-id-4471/ does not.
- Consistent templates. Predictable page shapes are easier for machines and calmer for humans.
None of this is an AI tactic. It is site architecture, and it has been good practice for fifteen years. AI search raised the penalty for skipping it. Our piece on AI search as a layer on top of SEO covers why the two are one job.
10-Point AI Readability Self-Check
Run this against your own site. Every “no” is a place a machine loses the thread.
| # | Question | Yes / No |
| 1 | Is every important page indexable, with nothing valuable behind a login or a noindex tag? | ☐ |
| 2 | Does each page have one clear H1 that states what the page is? | ☐ |
| 3 | Are headings organized logically, with H2s and H3s in order? | ☐ |
| 4 | Does your key content appear in the page source, rather than loading through JavaScript? | ☐ |
| 5 | Is structured data implemented where appropriate (organization, service, FAQ)? | ☐ |
| 6 | Is important content available as HTML text rather than only in images or PDFs? | ☐ |
| 7 | Does every service you sell have its own dedicated page? | ☐ |
| 8 | Do internal links connect related topics to each other? | ☐ |
| 9 | Is navigation simple, consistent, and shallow? | ☐ |
| 10 | Is the site mobile-friendly, fast, and free of blocked AI crawlers in robots.txt? | ☐ |
Score under eight and llms.txt is not your problem. Structure is.
Number four takes ten seconds to test. Open the page, press Ctrl+U to view the source, then search for a sentence you can see on screen. If it is missing from the source, an AI crawler cannot see it either.
What We Check When Building AI-Friendly Websites
No lever guarantees an AI citation, and any agency promising one is guessing. What a build can do is remove every reason a machine would fail to understand you.
On a Verta website build, that means:
- Crawlability first. robots.txt reviewed, sitemaps clean, AI crawler access deliberate rather than accidental.
- Server-rendered content. The words sit in the HTML before any script runs.
- Site architecture. A page per service, a flat hierarchy, and internal links that reflect how the business works.
- Entity clarity. One consistent name, address, and description across the site, the schema, and your business profiles.
- Structured data applied where it describes something real.
- Content hierarchy with honest headings that match what follows them.
- Core Web Vitals and accessibility, which serve people first and machines second.
That technical work sits alongside SEO and AI visibility, AEO and LLM optimization where it applies, and the content that gives a machine something worth quoting. A perfectly built site with nothing to say gets read and ignored.
The Takeaway
AI systems are not reading your website with alien intelligence. They read it quickly and literally, with no JavaScript and no patience. Everything that helps them is something a good developer would have done anyway.
So skip the trend chase. Get your content into HTML, give every service a page, keep your headings honest, and make the site fast. Publish an llms.txt if it takes an hour, then leave it alone until the evidence changes.
The businesses cited in AI answers over the next few years will mostly be the ones that were easy to understand all along. That foundation pays for itself in both kinds of search.
