# Allow OpenAI (ChatGPT, GPTBot) User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / # Allow Anthropic Claude User-agent: ClaudeBot Allow: / # Allow Google AI (Gemini / Bard) User-agent: Google-Extended Allow: / # Allow Amazon's AI bot User-agent: Amazonbot Allow: / # Allow Common Crawl (used by many AI orgs) User-agent: CCBot Allow: / # Allow Perplexity User-agent: PerplexityBot Allow: / # Allow You.com AI User-agent: YouBot Allow: / # Allow Neeva / Snowflake User-agent: NeevaBot Allow: / # Default allow all others User-agent: * Allow: /