Why does ChatGPT never mention us? What actually works for visibility in AI search

If an AI assistant answers questions about your industry and never mentions you, the cause is usually not weak content. It is one of three things: you are blocking the wrong bot without knowing it, your page does not answer the question directly, or your brand name barely appears anywhere else on the web. There is no separate AI file, no special markup and no hidden AEO setting. Google says as much in its own documentation. What is left is getting crawl permissions right, structuring pages so they answer, and being mentioned outside your own domain.

First question: does this traffic matter yet

By volume, it is still small. Independent measurements put referrals from standalone AI assistants at a fraction of a percent of total web traffic. Two things change that picture.

The first is behaviour on the results page itself. Pew Research Center tracked the browsing activity of 900 US adults during March 2025 across 68,879 Google searches. When an AI summary appeared, users clicked a traditional search result in 8% of visits. Without one, 15%. Clicks on links inside the summary happened in just 1% of visits. Same query, roughly half the clicks.

The second is the quality of what does come through. Semrush's data puts the average AI search visitor at 4.4 times the value of an average organic visit when measured by conversion rate. That tracks with how people use these tools. They do most of the research with the assistant and land on you at the decision stage. In B2B the gap widens, because the comparison and shortlisting work has already been done for them.

Three separate bots, three separate decisions

This is where most sites get it wrong. An AI company does not run one crawler. It runs at least three, doing different jobs, and you can address each one separately in robots.txt.

At OpenAI: GPTBot crawls content that may be used to train its foundation models. OAI-SearchBot exists to surface websites in ChatGPT's search features. ChatGPT-User fetches a page live when a user asks something, and OpenAI notes that because that action is user-initiated, robots.txt rules may not apply to it.

Anthropic mirrors the same split: ClaudeBot for training, Claude-SearchBot for the quality of Claude's search results, and Claude-User for retrieving a page in response to a user's question.

The practical consequence: if you do not want your archive training a model but you do want to appear in ChatGPT's search results, those are two different lines. Ticking a single "block AI bots" box closes both doors.

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Claude-SearchBot
Allow: /

The most common cause we find: you blocked yourself

When a client asks why their site never gets cited, robots.txt and the CDN bot rules are the first place we look. More often than not, that is where the answer is.

On 1 July 2025 Cloudflare began blocking AI crawlers by default for newly registered domains, moving the whole model to permission-first. As a deliberate choice that is defensible. Inherited by accident, it means ChatGPT or Perplexity never reach your site when they go looking for sources. The same thing happens through managed bot rules and WAF profiles, where a ready-made "block known bots" rule quietly catches the search crawlers too.

Checking takes five minutes. Search the last 30 days of server or CDN logs for OAI-SearchBot, Claude-SearchBot, PerplexityBot and Googlebot. If there are no hits, or every hit returns 403, your content is not the problem. The door is shut.

Settling the llms.txt question

The llms.txt idea, proposed in late 2024, is appealingly simple: a Markdown file at your site root that summarizes the site for language models. Marketing quickly framed it as the sitemap of AI SEO. The evidence does not support that framing.

In June 2026 Google added a clarification to its Search Central documentation stating that Google Search does not use llms.txt files, and that they produce neither a positive nor a negative effect. SE Ranking looked at roughly 300,000 domains, found the file on 10.13% of them, and found no measurable link between having it and how often models cited that domain. Dropping the variable from their model actually improved its accuracy.

That does not make the file pointless. Stripe, Vercel, Cloudflare and Anthropic all publish one, but for a different audience: feeding clean documentation to coding assistants and agents. If you ship developer docs, it is a reasonable investment. Adding it to a corporate marketing site and expecting search visibility buys you nothing.

What Google actually says, stripped down

Google's "AI features and your website" page cuts through most of the noise. To be eligible as a supporting link in AI Overviews or AI Mode, a page needs to be indexed and eligible to appear in Search with a snippet. There are no additional technical requirements. The documentation states it plainly: you do not need to create new machine readable files, AI text files or markup, and there is no special schema.org structured data you need to add.

Structured data still helps, just not for the reason people assume. Markup that matches the visible text on the page clarifies what the content is and makes you eligible for rich results. Stuffing invisible information into markup is a policy violation.

If you want to limit visibility, the levers live in the same place: nosnippet, data-nosnippet, max-snippet and noindex. Google has documented that the nosnippet and max-snippet rules also apply to its AI-powered search features. Suppressing your snippet keeps you out of AI Overviews, and it cuts your classic search presence at the same time. One switch, both outcomes.

Ranking on page one is no longer enough on its own

A year ago the advice was straightforward: rank in the top 10 and you get cited. The data has loosened that link considerably.

In a study published on 2 March 2026, Ahrefs analysed 863,000 keyword SERPs and 4 million AI Overview URLs. Only 37.9% of cited pages ranked in the top 10 for the same query. Another 31.2% sat between positions 11 and 100, and 31.0% fell outside the top 100 entirely. The same team's July 2025 measurement had put that first figure at 76%. It halved in eight months.

One detail is worth noting on its own: YouTube accounts for 5.6% of all AI Overview citations, and 18.2% of the citations that come from pages not ranking at all. A demo video explaining your product can end up being a more reachable source than your service page for certain queries.

Brand mentions correlate more strongly than backlinks

Ahrefs studied 75,000 brands with a Domain Rating above 40 and found the strongest signals for AI Overview visibility were all off-site. Brand mentions across the web showed a Spearman correlation of 0.664. Backlink counts came in at 0.218. Branded search volume and brand names appearing in anchor text both ranked ahead of backlink counts as well.

Correlation is not causation, and that caveat matters here. But the mechanism is plausible given how these models work. A model learns a brand from the words that surround its name, not from a link graph, which is why an unlinked mention still carries signal. In practice this means getting named in industry roundups, forum threads, comparison lists and customer references does more for you than a link from a directory site.

What a cited page looks like

No magic here, but there is discipline. Pages that get cited share a few habits.

The answer comes in the first paragraph. A page that buries the answer under the fifth heading cannot compete with one that delivers it in the first 60 words. One clear question per page beats a catch-all: pages built around "how long does X take" or "what is the difference between Y and Z" get cited, generic "our services" pages do not.

Headings use the same language as the question. Concrete details like numbers, dates, durations and price ranges go into a table or a list instead of dissolving into prose. Publication and update dates are visible, and it is clear who wrote the thing.

Then the technical half: the content has to be in server-rendered HTML. Traditional search crawlers execute JavaScript, but most AI crawlers take the raw HTML and move on. A pricing table or FAQ section that only draws on the client is content those bots never see.

Crawl or train, two different answers

These are separate decisions and they get expressed in separate places. In September 2025 Cloudflare introduced the Content Signals Policy as a robots.txt extension: a single line in the form Content-Signal: search=yes, ai-train=no lets you state your preference for search, for being used as input to AI answers, and for model training independently. For customers on Cloudflare's managed robots.txt, the default allows search and blocks training, while deliberately leaving the AI input preference unset rather than assuming it for you.

The important caveat: these are stated preferences, not technical controls. They stop nothing on their own. Real enforcement needs a WAF and bot management layer behind them. The statement still has value, because it puts your intent on record if a dispute ever arises.

You cannot manage what you do not measure

AI visibility shows up badly in conventional analytics, because being named in an assistant's answer produces no click at all. At minimum, set up three things.

Create a referral segment in analytics based on referring hostname: chatgpt.com, perplexity.ai, gemini.google.com, copilot.microsoft.com and claude.ai. Watch that segment's conversion rate next to your organic segment, because the volume is small but the quality difference shows up right there.

Build a per-bot report from your server logs. Which AI crawler pulls which pages and how often is the most direct evidence you have about what is entering the source pool at all.

Finally, write down 20 to 30 real questions about your category and ask the same assistants once a month, saving the answers. This manual tracking answers "am I visible" more honestly than most tools on the market. Looking at which competitor pages get named in those answers is instructive in its own right.

Four things to do this week

Open your robots.txt and CDN bot rules and split search crawlers from training crawlers into separate lines. Check the last 30 days of logs to confirm AI crawlers are actually reaching the site. For your three highest-revenue services, turn the concrete questions customers ask into headings and move the answer into the first paragraph. Create the AI referral segment in analytics so you have something to compare against three months from now.

At Wedevit we do not treat this as separate from technical SEO, because it is not a separate discipline: crawl permissions, rendering behaviour, page structure and measurement are all part of the same job. If you want to know where your site stands in AI search, we start with a one-off visibility and crawl audit. The whole engagement runs remotely.