Choose which AI crawlers can read your site, separately for search, training and user-requested visits, and get a robots.txt block you can paste in. Nothing you enter leaves your browser.
For most businesses that want to be named in AI answers, allow the search crawlers and decide separately about training. Search crawlers such as OAI-SearchBot, Claude-SearchBot and PerplexityBot build the indexes that AI search answers draw on, so blocking them removes you from those answers. Training crawlers such as GPTBot and ClaudeBot collect content that may be used to train future models; blocking them does not remove you from search, so it is a business decision rather than a visibility one.
Token
Operator
Used for
OAI-SearchBot
OpenAI
ChatGPT search results
GPTBot
OpenAI
Content that may train OpenAI models
ChatGPT-User
OpenAI
Pages fetched when a ChatGPT user asks
Claude-SearchBot
Anthropic
Search results quality for Claude
ClaudeBot
Anthropic
Content that may train Claude models
Claude-User
Anthropic
Pages fetched when a Claude user asks
PerplexityBot
Perplexity
Perplexity search and answers
Perplexity-User
Perplexity
Pages fetched when a Perplexity user asks
Google-Extended
Google
Gemini training and some grounding; not Google Search
Pick a preset, then adjust any crawler individually.
Paste your current robots.txt into “Your existing rules” so nothing you already block is lost.
Copy or download the result and replace the file at https://yourdomain/robots.txt.
Check your server or firewall is not blocking the same crawlers. A robots.txt allow cannot override a firewall block.
Allow about a day: OpenAI notes that search-related robots.txt changes can take around 24 hours to take effect.
Questions about AI crawlers
Will blocking GPTBot remove my site from ChatGPT?
No. GPTBot collects content that may be used for training. Appearing in ChatGPT search is controlled by OAI-SearchBot, which OpenAI recommends allowing. A site can block GPTBot and still be eligible for ChatGPT search answers.
Does Google-Extended affect AI Overviews?
No. AI Overviews and AI Mode are part of Google Search and use Googlebot. Google-Extended governs whether content can be used for Gemini training and some grounding, and blocking it does not change Google Search inclusion. To limit what Google shows from a page, use snippet controls such as nosnippet.
Do user-triggered fetchers obey robots.txt?
Not always. OpenAI says robots.txt rules may not apply to ChatGPT-User because it acts on a user's request rather than crawling automatically. Anthropic says its bots, including Claude-User, honour robots.txt. If you must block a fetcher reliably, do it at your server or firewall.
Is my data sent anywhere?
No. The generator runs entirely in your browser. Nothing you type is uploaded or stored.
What this means for you
See what the assistants say about you.
The free visibility sweep puts forty questions a real client would ask to five answer engines and records every answer and cited source, with three competitors on the same run.