Back to Crawlers & Access
Crawlers & Access

How do I allow OpenAI's search crawler in robots.txt?

Updated January 12, 2026GEO Expert Team

Summary

Add an allow rule for OAI-SearchBot and verify your robots.txt syntax. OpenAI recommends allowing OAI-SearchBot for inclusion in ChatGPT search results.

Detailed Answer

Add an allow rule for `OAI-SearchBot` (and verify your robots.txt syntax). OpenAI recommends allowing OAI-SearchBot for inclusion in ChatGPT search results.

Example robots.txt:
```
# Allow ChatGPT search visibility
User-agent: OAI-SearchBot
Allow: /

# Optionally block training
User-agent: GPTBot
Disallow: /
```

Verification steps:
1. Test robots.txt syntax
2. Check for conflicting rules
3. Monitor crawl stats (if available)
4. Review OpenAI's documentation for updates

openairobots-txtoai-searchbotconfiguration