GPTBot

GPTBot is OpenAI's web crawler that collects pages that may be used to train future GPT models. It's separate from the bots ChatGPT uses to search.

TechnicalUpdated 1 min read

Also called OpenAI crawler · OpenAI bots

What it is

OpenAI announced GPTBot in 2023. It crawls public pages that may be used to train future models. It follows robots.txt.

OpenAI's three bots

BotWhat it doesBlock it and...
GPTBotCollects pages for trainingFuture models learn less from your site
OAI-SearchBotIndexes pages for ChatGPT searchYour pages can't show up as ChatGPT sources
ChatGPT-UserVisits a page when a user's chat needs itChatGPT can't open your page live

Which to allow

If you want ChatGPT to recommend you and link to you, allow OAI-SearchBot and ChatGPT-User. GPTBot is a separate choice about training. Most businesses that want to be known allow all three.

Example

A SaaS company blocked "GPTBot" and, by mistake, "OAI-SearchBot" in the same rule. Its pages never appear in ChatGPT's sources. After allowing OAI-SearchBot, they start appearing within weeks.

Free tool: AI Crawler CheckerSee which AI crawlers your robots.txt and firewall let in, and the exact line that blocks each one.Try it →

Frequently asked questions

Does blocking GPTBot remove me from ChatGPT?

No. ChatGPT can still search and cite you if OAI-SearchBot is allowed, and other sites still mention you.

How do I know GPTBot visits my site?

Check your server logs for "GPTBot" in the user agent. See AI crawler logs.

Can fake bots pretend to be GPTBot?

Yes. OpenAI publishes the IP ranges its bots use, so you can check.

Go deeper

All posts →

Make AI search your next revenue channel

Start free trial