AI crawler log analyzer
Paste a small sample of nginx, Apache, Cloudflare, or JSONL access logs. The analyzer looks for recognizable search and AI crawler user agents, checks whether core discovery files were requested, and produces a copyable proof report.
Paste access logs
Works best with combined log format or JSON lines containing fields such as path, status, userAgent, source, or referrer. Keep samples small and remove private query strings before sharing reports.
Copyable report
Crawler summary
Detected crawlers
| Crawler | Hits | Status mix | Important paths |
|---|
Recommendations
Source-backed notes
- Google documents Googlebot as the crawler affecting Google Search, while
Google-Extendedis a separate product token and not a Search ranking signal. Google crawler docs - OpenAI documents
OAI-SearchBot,GPTBot, and user-triggered fetchers separately. OpenAI crawler docs - Apple, Perplexity, and Common Crawl publish crawler names that can be checked in logs and robots.txt policies. Applebot, Perplexity crawlers, CCBot