PerplexityBot vs Perplexity-User: How Perplexity Fetches Content

The simple version of PerplexityBot vs Perplexity-User is tempting: one crawls the web in the background, while the other fetches pages for someone asking a question right now.

That is probably the most useful working model. But it is not quite an official definition.

Perplexity’s own documentation names both identifiers, gives them separate IP-range endpoints, and distinguishes automatic collection from collection in response to user requests. It does not map every step of discovery, indexing, and live retrieval to one agent or the other. The neat division of labor comes mainly from third-party descriptions, and those descriptions disagree at one important point: which agent performs a fresh query-time fetch.

So the real question is not merely what the two names seem to mean. It is what a request carrying either name actually lets you conclude.

The documented distinction: two identifiers, incomplete role definitions

Perplexity’s crawler documentation says it uses web crawlers and user agents to gather and index information from the internet, either automatically or in response to user requests. This gives us two broad request contexts, but the passage describes the tools collectively. It does not say, in that statement, that PerplexityBot always handles automatic work and Perplexity-User always handles user-triggered work.

The documentation does make a narrower distinction. It lists PerplexityBot and Perplexity-User as separate values and publishes a different official IP endpoint for each:

• https://www.perplexity.com/perplexitybot.json
• https://www.perplexity.com/perplexity-user.json

This matters because Perplexity treats the two as independently identifiable traffic for access-control purposes. Its WAF guidance says to create separate IP sets and combine each relevant User-Agent match with the corresponding IP range.

That is the firm baseline: there are two declared identifiers, and Perplexity expects operators to distinguish them.

What the official material does not give us is a complete semantic definition of that distinction. It does not establish whether the names differ strictly by crawler role, by the context that caused a request, by some form of authorization, or by another internal rule. Nor do the supplied official passages give exact literal HTTP User-Agent headers for both agents, complete versioning rules, or a full authentication protocol.

There is a reported full string for Perplexity-User, but its source matters. In an independent investigation, Cloudflare recorded this browser-compatible form:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)

That is a third-party observation, not the equivalent of an official promise that the string will always have that exact form. The supplied evidence also does not give a matching official full string for PerplexityBot.

This leaves us with an odd but useful result. The two-agent distinction is official; the complete account of what distinguishes them is not.

From discovery to retrieval: the best-supported content-fetching model

It helps to split “finding a page” into separate things, because otherwise crawling, search, and fetching blur together.

First comes candidate discovery. A search system needs some way to identify pages that may answer a query. That may draw on an existing index—a stored representation of pages previously found and processed—and may produce candidates with metadata rather than freshly reading every page at that moment.

Perplexity’s Search API documentation supports this distinction. It describes web search with ranked results, domain filtering, and content extraction. The API reference says the API can search the web and retrieve relevant page contents; it can return structured fields including a title, URL, snippet, date, and last-updated value. Domain filters can narrow the set of candidates considered.

A result URL or snippet therefore does not, by itself, prove that the full page was fetched during that particular search. Candidate discovery and page retrieval are related, but they are not the same event.

The commonly described Perplexity indexing and crawling model looks like this: PerplexityBot performs broad, automatic crawling to build or refresh an index; Perplexity-User visits selected pages when the assistant browses in response to a live question. XSeek describes PerplexityBot as the general-purpose indexing crawler and Perplexity-User as the real-time agent for current user queries. 51Degrees likewise describes PerplexityBot as a general web crawler and Perplexity-User as being triggered when someone asks Perplexity a question.

This model fits the two broad contexts in Perplexity’s official documentation. It is also intuitively clean: one system learns what is out there; another goes back to read a promising page when freshness matters.

But this is where the apparent clarity starts to break down.

A separate third-party workflow description says a user query may lead Perplexity to identify candidate pages from an index, freshly fetch leading candidates for time-sensitive or high-confidence work, parse the retrieved content, extract useful passages, synthesize an answer, and attach citations. This is a plausible sequence, and it usefully separates search from retrieval and later processing. It is not, however, a first-party description of Perplexity’s production pipeline.

More importantly, the supplied third-party sources do not agree on the agent responsible for the fresh fetch. One assigns live browsing to Perplexity-User; another describes fresh query-time fetching in an account centered on PerplexityBot. Perplexity’s official passages do not settle the dispute.

So the best-supported model has two levels of confidence. It is well supported that Perplexity does both automatic collection and collection in response to user requests, and that discovery can be distinguished from retrieval. It is reasonably supported—but by third parties—that PerplexityBot is associated with index building and Perplexity-User with live browsing. It is not established which agent performs every fresh page request, or whether the same assignment holds across every Perplexity product and query.

Triggers and operational behavior: what requests can—and cannot—show

Suppose Perplexity-User appears in your server log. Does that prove someone asked a question that caused Perplexity to select your page?

It supports that interpretation under the common third-party model. It does not prove it.

A server log can show the URL requested, the request time, the source IP, and the User-Agent string the client chose to present. If the claimed agent and source address also match Perplexity’s current official range, the request fits Perplexity’s documented pattern for declared traffic. None of this exposes the internal event that caused the request.

You cannot infer from that log entry alone whether the request arose from a scheduled crawl, a live question, a retry, or another service. Nor can you prove that the fetched page was indexed, used in an answer, quoted, cited, or even parsed successfully. A PerplexityBot request similarly shows access by traffic claiming that identity; it does not reveal the state of an index or guarantee that the page will become a candidate later.

This distinction is easy to miss because the names sound like explanations. They are really observations at the HTTP boundary.

The official documentation also says nothing precise about crawl schedules, recrawl intervals, freshness windows, concurrency, page-selection rules, or requests per second. It says the official IP ranges are updated regularly, but that is an infrastructure fact, not a crawl-frequency guarantee.

There are outside measurements, though they should stay in their proper box. A 2026 CaptainDNS report observed PerplexityBot making fewer than one request per second on public sites and reported occasional Perplexity-User spikes during user visits. The same report says these numbers are not publisher guarantees and may change. They are observations useful for context, not limits on which an operator should rely.

Thus a request can answer “what reached my server?” fairly well. It is much worse at answering “why did Perplexity send this, and what happened afterward?” The second question belongs to an internal pipeline that the supplied documentation does not expose.

Robots.txt, WAFs, and verification of declared requests

Perplexity presents robots.txt as a way for webmasters to manage how their sites and content interact with its crawlers. It also treats PerplexityBot and Perplexity-User separately when explaining WAF configuration. But separate configuration does not necessarily mean separate robots.txt policy: the supplied official passages do not state different directives or compliance rules for the two agents.

For declared requests, Perplexity’s documented WAF approach is straightforward:

1. Create separate IP sets for PerplexityBot and Perplexity-User from their respective official JSON endpoints.
2. Create a rule that combines the relevant User-Agent string with its corresponding IP set.
3. Set an allow action where that is the intended policy.
4. Give the allow rule priority over conflicting blocking rules.
5. Keep the IP sets current, because Perplexity says the published ranges are updated regularly.

This is stronger than trusting a name in a header. A User-Agent is just text supplied by the client and can be spoofed. An IP match alone is also incomplete because it does not state which agent the request claims to be, and ranges can change. Perplexity therefore recommends using both signals together.

Even both signals together are not cryptographic authentication. The official material does not document request signing, a complete challenge process, or an official reverse-DNS procedure. XSeek suggests reverse DNS as an additional check, but that is third-party guidance rather than a requirement or guarantee established by the supplied Perplexity documentation.

Robots.txt is different again. It expresses a site’s crawling preferences; it does not authenticate the client and is not itself a network barrier. A WAF can allow, block, or challenge an HTTP request, while robots.txt tells a cooperative crawler which paths it should access. Treating either one as a substitute for the other confuses policy with enforcement.

Timing adds another caveat. Perplexity says robots.txt setting changes may take up to 24 hours to appear in its systems. Its WAF guidance also warns that configuration changes may take time to propagate and recommends monitoring logs. Neither statement guarantees that every request will stop or begin immediately, or that an allowed request will result in a successful fetch.

The practical question, then, is narrower than “Can I identify Perplexity?” You can identify traffic consistent with Perplexity’s declared-agent scheme by checking the claimed agent against the corresponding current official IP range. You cannot turn that match into proof of the request’s internal purpose, and you cannot turn a non-match into proof that the request has no connection to Perplexity.

The unresolved edge: declared agents versus alleged undeclared traffic

The two-agent model explains declared traffic. It does not conclusively explain every request that others have associated with Perplexity.

Cloudflare reported that some customers blocked both declared agents with WAF rules and disallowed crawling in robots.txt, yet Perplexity still appeared able to access the relevant content. In Cloudflare’s own tests, it said it observed browser-like User-Agent strings, IP addresses outside Perplexity’s published ranges, and requests moving across IPs and autonomous system numbers, or ASNs—the network identifiers used in internet routing.

Cloudflare characterized this as stealth or undeclared crawling. It also reported behavior that ignored, or sometimes did not fetch, robots.txt. When Cloudflare said it successfully blocked the traffic it had identified, it observed Perplexity turning to other sources and producing answers that lacked details from the blocked page.

These are serious observations, but they are still observations and an attribution by Cloudflare. They are not Perplexity’s definition of PerplexityBot or Perplexity-User, and the supplied evidence does not independently prove that the traffic was run directly by Perplexity.

There is a competing account. The third-party guide by Sunil Pratap Singh reports that Perplexity distinguished its own indexing system from third-party model-training providers and attributed identified stealth visits to those providers. The supplied material does not include a direct first-party response that resolves the specific technical observations, so the conflict remains open.

Nor can the reports establish universal or current behavior. Cloudflare described what it saw in its tests and network telemetry, and it expected crawler behavior to evolve after publication. Another third-party account says reports had decreased by 2025 while arguing that generic requests from cloud infrastructure remained technically possible. Neither amounts to a systematic, current measurement of all Perplexity-associated fetching across the web.

This is the reveal in the PerplexityBot vs Perplexity-User distinction: the hard part is not telling the two declared names apart. Perplexity has supplied enough information to do that. The hard part is deciding what universe those names cover.

For traffic that keeps a declared identity, the operating model is useful. PerplexityBot is best treated as associated with general crawling and index refreshes; Perplexity-User is best treated as associated with retrieval prompted by live questions. Match the name with its corresponding current official IP range, and regard the result as evidence of declared Perplexity traffic—not as proof of the hidden trigger or later use.

For traffic outside that scheme, the evidence supports no clean label. A generic browser header can be used by almost anything. An out-of-range IP does not prove Perplexity ownership. Cloudflare’s findings justify treating robots.txt and declared-agent rules as controls with a defined scope, while the competing attribution prevents those findings from becoming a settled claim that Perplexity itself operated every observed request.

The precise question for an operator is therefore not “Have I allowed or blocked Perplexity in every possible form?” The evidence cannot promise that. It is: “Which declared requests can I verify, which requests merely claim an identity, and what will I do with the ones the two-agent model cannot explain?”