AI crawler directory / ChatGPT-User
ChatGPT-User
Documented Provider-maintained crawler reference.
Content ID: OFB-C006
ChatGPT-User identifies retrieval performed in response to a user's action in ChatGPT. It is separate from OpenAI's background search crawler and model-training crawler, so it should be evaluated as its own access decision.
Reference facts
- Provider
- OpenAI
- robots.txt token
ChatGPT-User- Purpose
- User-triggered retrieval
- Crawls pages
- Yes
- Obeys robots.txt
- Yes — documented by provider
- Search-visibility relevance
- Indirect
- Status
- active
- Last verified
- Revision
- 1.0
What it controls
- Whether matching public URLs may be retrieved during supported user-initiated ChatGPT workflows.
- A policy choice about interactive access rather than background indexing or training.
What it does not control
- OAI-SearchBot discovery or GPTBot model-training crawling.
- Whether ChatGPT will answer a question, cite the page, or use a third-party search result.
- Access to content protected by authentication or other real security controls.
Recommended policy notes
- Evaluate ChatGPT-User as a user-triggered retrieval policy, not as background search indexing or model training.
- Decide whether users should be able to ask ChatGPT to retrieve public pages from the site.
Example policies
Allow user-triggered retrieval
User-agent: ChatGPT-User
Allow: /Keep sensitive or private content behind authentication; robots.txt is not an access-control mechanism.
Record the current policy without recommending a change
# Review the existing ChatGPT-User group and document the intentional policy.Common misconceptions
- ChatGPT-User is the same crawler as GPTBot or OAI-SearchBot.
- Allowing ChatGPT-User guarantees that every user-requested fetch will succeed.
How OpenForBots evaluates it
- Parses the robots.txt rules that apply to ChatGPT-User and reports the matching directive.
- Labels the result separately from OpenAI search and training findings.
- Does not impersonate a real user or claim that a simulated request reproduces a production ChatGPT workflow.
Limitations
- A robots.txt evaluation cannot reproduce every ChatGPT product path or user interaction.
- A user-agent header can be spoofed and does not authenticate a requester.
- Network, WAF, consent, authentication, and product-level restrictions may still affect retrieval.