AI crawler
Perplexity-User
Perplexity's user-requested fetch crawler.
- Vendor
- Perplexity
- robots.txt token
Perplexity-User- Purpose
- User-requested fetch
- Respects robots.txt
- Generally no. Perplexity says it generally ignores robots.txt rules
- Source
- Vendor documentation
What it does
Perplexity says Perplexity-User "supports user actions within Perplexity".
Because the fetch is requested by a user, Perplexity says it generally ignores robots.txt rules.
What blocking it changes
A robots.txt block for Perplexity-User is a statement of intent, not an enforced outcome. The vendor documentation below says these fetches may not follow robots.txt, because a user asked for the page.
robots.txt
Block it
User-agent: Perplexity-User
Disallow: /Allow it
User-agent: Perplexity-User
Allow: /Other crawlers from Perplexity
- PerplexityBot (search)
Check your own site
The free Chompr.io extension shows each AI search crawler's status for the page you are on.