AI crawler
ChatGPT-User
OpenAI's user-requested fetch crawler.
- Vendor
- OpenAI
- robots.txt token
ChatGPT-User- Purpose
- User-requested fetch
- Respects robots.txt
- Not necessarily. OpenAI says robots.txt rules may not apply to user-initiated actions
- Source
- Vendor documentation
What it does
OpenAI says ChatGPT-User visits pages when a user asks ChatGPT or a Custom GPT a question, or uses GPT Actions.
It says ChatGPT-User "is not used for crawling the web in an automatic fashion" and is not used to determine search inclusion, which OAI-SearchBot governs.
What blocking it changes
A robots.txt block for ChatGPT-User is a statement of intent, not an enforced outcome. The vendor documentation below says these fetches may not follow robots.txt, because a user asked for the page.
robots.txt
Block it
User-agent: ChatGPT-User
Disallow: /Allow it
User-agent: ChatGPT-User
Allow: /Other crawlers from OpenAI
- OAI-SearchBot (search)
- GPTBot (training)
Check your own site
The free Chompr.io extension shows each AI search crawler's status for the page you are on.