What we can check on each AI search platform
We test ChatGPT Search, the Gemini app, Google Search (including AI Overviews and AI Mode), Claude web search, Perplexity and Microsoft Copilot. Each observation names the exact surface, mode and date, because each platform shows sources differently and uses different crawlers.
| Surface | When sources are visible | Crawler controls | What we record |
|---|---|---|---|
| When ChatGPT searches the web, answers show source links you can open. | OAI-SearchBot (search). GPTBot is the separate training crawler; ChatGPT-User fetches pages a user asks for. | Question, search on or off, date, location setting, names in the answer, cited URLs. | |
| Some Gemini answers show links; many do not. We record whether any links were shown. | Google-Extended is a separate product control for certain Gemini training and grounding uses. It does not affect Google Search inclusion. | Question, app or web, date, whether links were shown, names in the answer. | |
| AI Overviews and AI Mode show links. Pages need to be indexed and eligible for a snippet to appear as a link. | Googlebot. No special AI file or markup is needed; standard Search guidance applies. | Query, whether an AI feature appeared, date, location, linked pages. With access: Search Console’s Generative AI performance report. | |
| When Claude searches the web, answers include citations to the pages used. | Claude-SearchBot (search) and Claude-User (user-requested fetches). ClaudeBot is the separate training crawler. | Question, web search on, date, names in the answer, cited URLs. | |
| Answers are normally shown with numbered source links. | PerplexityBot (search discovery). Perplexity-User fetches pages for a user’s question and generally ignores robots.txt. | Question, mode, date, names in the answer, each numbered source. | |
| Answers grounded in Bing results show citations. | Bingbot. With access: Bing Webmaster Tools AI Performance report (citation counts, referenced pages, grounding queries). | Question, date, names in the answer, cited URLs. Bing citation totals are counts, not ranks. |
ChatGPT Search
When ChatGPT searches the web, answers show source links you can open. OpenAI says opted-out sites will not be shown in ChatGPT search answers, and that robots.txt changes can take about 24 hours to apply.
Crawlers: OAI-SearchBot (search). GPTBot is the separate training crawler; ChatGPT-User fetches pages a user asks for.
How we record it: Question, search on or off, date, location setting, names in the answer, cited URLs.
Gemini app
Some Gemini answers show links; many do not. We record whether any links were shown. Gemini app results are never reported as Google Search results, or the other way round.
Crawlers: Google-Extended is a separate product control for certain Gemini training and grounding uses. It does not affect Google Search inclusion.
How we record it: Question, app or web, date, whether links were shown, names in the answer.
Google Search, including AI Overviews and AI Mode
AI Overviews and AI Mode show links. Pages need to be indexed and eligible for a snippet to appear as a link. AI features do not appear for every query. When none is shown, we record that.
Crawlers: Googlebot. No special AI file or markup is needed; standard Search guidance applies.
How we record it: Query, whether an AI feature appeared, date, location, linked pages. With access: Search Console’s Generative AI performance report.
Claude web search
When Claude searches the web, answers include citations to the pages used. Blocking Claude-SearchBot or Claude-User may reduce how visible and accurate your site is in those results.
Crawlers: Claude-SearchBot (search) and Claude-User (user-requested fetches). ClaudeBot is the separate training crawler.
How we record it: Question, web search on, date, names in the answer, cited URLs.
Perplexity
Answers are normally shown with numbered source links. Perplexity publishes IP ranges so a CDN rule can verify its crawler instead of blocking it.
Crawlers: PerplexityBot (search discovery). Perplexity-User fetches pages for a user’s question and generally ignores robots.txt.
How we record it: Question, mode, date, names in the answer, each numbered source.
Microsoft Copilot
Answers grounded in Bing results show citations. IndexNow can notify Bing of substantive URL changes. It is a notification, not a guarantee of inclusion.
Crawlers: Bingbot. With access: Bing Webmaster Tools AI Performance report (citation counts, referenced pages, grounding queries).
How we record it: Question, date, names in the answer, cited URLs. Bing citation totals are counts, not ranks.
Primary sources
- OpenAI: Overview of OpenAI crawlers
- Google Search Central: AI features and your website
- Google: Common crawlers, including Google-Extended
- Anthropic: Crawlers and how site owners can control them
- Perplexity: Perplexity crawlers
- Bing Webmaster Blog: AI Performance in Bing Webmaster Tools (public preview)
- IndexNow: FAQ
Checked on 18 November 2026. Platform behaviour changes; we re-check these references when we revise the page.