What it is
Claude is an AI assistant and model family from Anthropic, an AI company founded in 2021. It's used heavily for writing, analysis and coding, and it's built into tools from other companies too.
Claude can search the web and cite sources, like ChatGPT.
Its crawlers
Anthropic runs separate bots, which you can allow or block in robots.txt:
- ClaudeBot: collects pages for training.
- Claude-SearchBot: indexes pages for search.
- Claude-User: fetches a page when a user's question needs it.
How much it matters
Claude has fewer consumer users than ChatGPT or Google's AI, but it's common among technical and business users. If you sell software or services to those buyers, it's worth tracking.
Example
A developer tool finds most of its signups mention "asked Claude" or "asked ChatGPT". It adds Claude to the engines it tracks and finds it's named there more often than on Gemini.
Frequently asked questions
Does blocking ClaudeBot remove me from Claude?
It stops new training crawls. Allow Claude-SearchBot and Claude-User if you want Claude to read your pages when answering.
Does Claude recommend products?
Yes, when asked. Like other AI tools, it relies on what it learned and on pages it reads.
Is Claude in other products?
Yes. Many apps use Claude models behind the scenes, often without naming it.