PerplexityBot
Perplexity's search crawler.
- Vendor
- Perplexity
- robots.txt token
PerplexityBot- Purpose
- Search
- Respects robots.txt
- Yes, according to Perplexity's documentation
- Source
- Vendor documentation
What it does
Perplexity says PerplexityBot is "designed to surface and link websites in search results on Perplexity".
It says the bot is "not used to crawl content for AI foundation models", and recommends allowing it in robots.txt if you want your site to appear in Perplexity search results.
What blocking it changes
PerplexityBot is how Perplexity's answer engine indexes pages, so blocking it removes your pages from that engine's answers.
Perplexity's documentation recommends allowing PerplexityBot if you want your site to appear in its search results, so blocking it works against that.
In the Tow Center (Columbia) generative-search study, which made 1,600 observations, pages from publishers that blocked an engine's crawler were cited about half as often (35.5% against 17.7%). OpenAI honoured blocks far more than Perplexity did. The data covered OpenAI and Perplexity only. The effect is not guaranteed for your own site. See the crawler evidence for more.
robots.txt
Block it
User-agent: PerplexityBot
Disallow: /Allow it
User-agent: PerplexityBot
Allow: /Other crawlers from Perplexity
- Perplexity-User (user-requested fetch)
Check your own site
The free Chompr.io extension shows each AI search crawler's status for the page you are on.