Why AI assistants give different answers to one question
By the Why Won't AI team · Published · Last updated
As of November 2026, each AI assistant finds sources with its own search system, chooses which sources to show with its own rules, and writes a new answer each time. The same question can name different products on ChatGPT, Gemini, Google AI Overviews, Claude and Perplexity, and can change from one day to the next. Track each assistant separately.
What makes the answers differ
- Different search systems. Each assistant draws on its own index or search partner, so a page can be known to one and not to another.
- Different crawlers. Each company uses its own crawlers and robots.txt names, and your site can allow one and block another.
- Different rules for sources. Some answers show many links, some show few, and Google shows an AI Overview only for some searches.
- The app is not the API. The answer a person sees in an app can differ from the answer the same model gives through its API.
- Answers change. The same question can give a different answer on a second run.
What our own test showed
On November 3, 2026 we asked 15 buyer questions in our category once on each assistant. This is how often each one named four of the tools in our market (out of 15 answers; Perplexity 14, because one request failed):
| Tool named | ChatGPT | Gemini | AI Overviews | Claude | Perplexity |
|---|---|---|---|---|---|
| Otterly.ai | 5 | 5 | 5 | 6 | 1 |
| Peec AI | 5 | 3 | 4 | 5 | 3 |
| Ahrefs | 3 | 3 | 3 | 3 | 3 |
| Profound | 3 | 3 | 2 | 4 | 2 |
- Perplexity linked to otterly.ai in 4 answers but named Otterly in only 1. A link and a mention are not the same thing.
- Perplexity answers carried the most links: from 15 to 43 per answer in our sample.
- ChatGPT linked to vendor sites more often than the other assistants did.
- YouTube was cited only in Google AI Overviews, and Reddit mostly in AI Overviews and Gemini.
This is one run of 15 questions, so treat it as an example of how far apart the assistants can be, not as a ranking.
What to do with this
- Report each assistant on its own line. One average hides the gaps.
- Check robots.txt and your CDN for each company's crawlers, not only Googlebot.
- Look at which kinds of sources each assistant cites, and be present where your buyers' assistant looks.
- Run the same questions on a schedule before you decide that an assistant ignores you.
Questions people ask
Which AI assistant matters most?
The one your buyers use. Ask customers, and check which assistants send visits in your analytics.
Why does Google AI Overviews sometimes give no answer?
Google shows an AI Overview only when it decides one adds value to the search. A search with no AI Overview is not a sign that you are missing.
Can one page work for every assistant?
A clear page that crawlers can read helps everywhere. Where you get cited still differs, because each assistant uses different sources.
What this does not tell you
Which assistant your own buyers use most, and which sources each assistant trusts in your category.
Sources
- OpenAI: Overview of OpenAI crawlers: what OAI-SearchBot, GPTBot and ChatGPT-User each do
- Google Search Central: AI features and your website: how AI Overviews and AI Mode choose pages