GPTBot vs. OAI-SearchBot vs. ChatGPT-User: which should you block?
Published 7/24/2026
OpenAI documents three distinct crawler tokens, each triggered by a different thing. Treating them as one “OpenAI bot” and blocking or allowing all three together is the most common misconfiguration CrawlPact sees in this operator’s traffic.
The three tokens
GPTBot— training. Crawls content that may inform future model training.OAI-SearchBot— search. Surfaces and links to pages in ChatGPT search results.ChatGPT-User— user-triggered. Fetches a specific page because a person asked ChatGPT a direct question about it.
The decision
- Disallow
GPTBotif you don’t want your content used for OpenAI’s model training. This has no effect on whether your site can be found through ChatGPT search. - Leave
OAI-SearchBotallowed if you want your pages discoverable through ChatGPT’s search features — disallowing it removes that visibility entirely, similar to disallowing a search engine’s indexer. - Leave
ChatGPT-Userallowed unless you specifically don’t want any content served in response to direct user questions inside ChatGPT — blocking it does not reduce training or search exposure, since it’s a separate, user-initiated request path.
Common mistake CrawlPact flags
A single User-agent: GPTBot group is sometimes assumed to cover “all of OpenAI.” It doesn’t —
robots.txt groups match on the exact token named, with no fallback to a shared vendor identity.
Each token needs its own explicit Disallow if you want to restrict it. CrawlPact’s
AI crawler checker shows the result for all three, evaluated
independently, so a gap is visible immediately.
Related guides
Applebot vs. Applebot-Extended: Search/Siri vs. Apple Intelligence
Apple separates its long-standing search crawler from a newer, generative-AI-specific opt-out token. A decision guide for telling them apart.
Blocking AI training while staying visible in AI search
Choosing between CrawlPact's presets when the goal is opting out of model training without losing AI-search discoverability.
ClaudeBot vs. Claude-User vs. Claude-SearchBot: which should you block?
Anthropic operates three separate crawler tokens for training, user-triggered retrieval, and search. A decision guide for configuring each independently.
See how this applies to your own site
Run a free audit to check your declared AI crawler policy against your own domain.
Audit a domain