AI crawler directory / PerplexityBot
PerplexityBot
Documented Provider-maintained crawler reference.
Content ID: OFB-C008
PerplexityBot is Perplexity's crawler for surfacing and linking websites in Perplexity search results. Perplexity documents that it is not used to crawl content for foundation-model training and publishes IP ranges that can help site owners verify legitimate requests.
Reference facts
- Provider
- Perplexity
- robots.txt token
PerplexityBot- Purpose
- Search and discovery
- Crawls pages
- Yes
- Obeys robots.txt
- Yes — documented by provider
- Search-visibility relevance
- Direct
- Status
- active
- Last verified
- Revision
- 1.0
What it controls
- Whether PerplexityBot may crawl matching URLs for Perplexity's search index.
- A technical access condition relevant to appearing as a linked source in Perplexity results.
What it does not control
- Perplexity-User requests initiated by a person using the product.
- Whether a page will be indexed, selected, quoted, ranked, or cited in an answer.
- Requests that only spoof the PerplexityBot user-agent string.
Recommended policy notes
- Allow PerplexityBot when the site's policy goal includes discoverability in Perplexity search results.
- Treat Perplexity-User separately because it represents user-requested fetches rather than background indexing.
Example policies
Allow Perplexity search discovery
User-agent: PerplexityBot
Allow: /Use WAF rules only after checking Perplexity's current IP-range publication.
Allow PerplexityBot
User-agent: PerplexityBot
Allow: /Common misconceptions
- Every Perplexity-attributed request is PerplexityBot.
- Allowing PerplexityBot guarantees that Perplexity will cite the site.
How OpenForBots evaluates it
- Parses robots.txt for the group that applies to PerplexityBot and records the matching rule.
- Links to Perplexity's current published IP ranges but does not claim that a robots.txt simulation verifies live traffic.
- Keeps background discovery separate from Perplexity-User retrieval.
Limitations
- Published IP ranges can change and must be treated as time-sensitive source data.
- OpenForBots does not verify that Perplexity has indexed a page or will use it in a response.
- robots.txt permission does not bypass authentication, WAF challenges, rate limits, or server failures.
Related guides and concepts
- AI Search Crawlers vs Training Crawlers
- Allow AI Search While Blocking Model Training
- How the OpenForBots Audit Works