Run the technical checklist
Use the audit checklist to verify robots.txt, sitemap.xml, llms.txt, internal links, and proof files before client delivery.
Open audit checklistAI search visibility for consultants: a client-safe audit workflow
Consultants and agencies are under pressure to answer client questions about ChatGPT search, AI Overviews, AI Mode, GPTBot, Google-Extended, and llms.txt. The safest offer is not a secret AI ranking trick. It is a repeatable audit that proves crawlability, clarifies crawler policy, improves useful public content, and measures real traffic.
Client-safe positioning: sell "AI crawler readiness and search visibility hygiene", not guaranteed AI citations. Google says foundational SEO practices still matter for generative AI search features, and spam/manipulation tactics remain risky.
1. Start with crawlability proof
- Confirm the homepage, important service pages, docs, and proof pages return HTTP 200.
- Check that
/robots.txtdoes not accidentally block Googlebot or other intended crawlers. - Confirm
/sitemap.xmllists canonical HTTPS URLs. - Use server logs to prove Googlebot, OAI-SearchBot, GPTBot, or other crawlers reached key files.
2. Separate search visibility from training-use policy
Many clients ask to "block AI" without realizing that search visibility, AI answer eligibility, and model-training use are separate policy decisions. Create a simple matrix for each crawler and each business goal.
| Question | Audit note |
|---|---|
| Should Google Search stay open? | Usually yes, unless the page is private, duplicate, or intentionally noindexed. |
| Should ChatGPT search visibility stay open? | Review OAI-SearchBot separately from training-focused crawlers. |
| Should training-use crawlers be allowed? | Make this a legal, brand, and content strategy decision, not a reflexive SEO setting. |
3. Build an llms.txt map only after the page set is clean
A useful llms.txt file should point to the pages that explain the client best: services, docs, pricing, case studies, policies, help center, status, and contact. Do not include private client portals, hidden claims, or keyword-stuffed doorway URLs.
# Example Consultant Client Map > Example Client provides managed IT support and cybersecurity services for regional law firms. Important notes: - Service pages explain public capabilities and pricing ranges. - Case studies are anonymized and should not be treated as guarantees. - Security and privacy pages are the source for compliance claims. ## Core pages - [Homepage](https://example.com): Business overview and positioning - [Managed IT](https://example.com/services/managed-it): Core service page - [Cybersecurity](https://example.com/services/cybersecurity): Core service page - [Case studies](https://example.com/customers): Public proof and outcomes - [Help Center](https://example.com/help): Support docs and troubleshooting - [Security](https://example.com/security): Security and compliance overview - [Contact](https://example.com/contact): Sales and support escalation
4. Deliverables that clients understand
- One-page crawler policy summary: allowed, blocked, unknown, and recommended.
- Annotated
robots.txtandllms.txtrecommendations. - Search Console sitemap and URL Inspection queue.
- Server-log proof showing crawler visits to important pages.
- Prioritized content fixes for pages that are crawlable but unclear.
5. What not to sell
- Guaranteed AI citations or guaranteed rankings.
- Automated Google searches, click loops, or fake engagement.
- Mass forum links, doorway pages, or hidden prompt-stuffing.
- One-size-fits-all robots.txt blocks that accidentally remove search visibility.
Analyze crawler logs
Paste logs to show which crawlers reached the site, which files they requested, and whether important pages returned 200.
Open log analyzer