65 articles
Built as a focused topical cluster around llms.txt, crawler policy, indexing, logs, distribution, and proof.
50 practical articles for AI crawler visibility and proof-based SEO
This library turns the first launch sprint into crawlable, internally linked, source-backed pages. Each article answers one narrow job and points to a related tool, checklist, template, or proof surface.
SEO setup included
Canonical URLs, Article schema, FAQ schema, sitemap entries, feeds, llms.txt references, and internal links.
No fake proof
Indexing requests and crawler logs are tracked as setup signals, not as traffic or ranking guarantees.
crawler policy
llms.txt vs robots.txt: what each file should do
A practical comparison of llms.txt and robots.txt for site owners who want AI crawler visibility without confusing access control.
Read the guideAI crawler user-agent cheat sheet
A compact cheat sheet for common AI and search crawler user-agent tokens and what to check before changing access.
Read the guideopenai crawlers
GPTBot vs OAI-SearchBot in plain English
A simple policy guide for deciding whether to allow GPTBot, OAI-SearchBot, or both in robots.txt.
Read the guidegoogle crawlers
Google-Extended vs Googlebot: do not block the wrong crawler
How to separate Google Search crawling from Google-Extended controls when writing an AI-era robots.txt policy.
Read the guidelaunch checklist
ChatGPT search visibility checklist for the first week
A seven-day checklist for making a new site easier to discover, crawl, and understand in ChatGPT-related search workflows.
Read the guideNew domain SEO day-one checklist
The day-one checklist for DNS, HTTPS, canonical URLs, sitemap, robots.txt, llms.txt, feeds, and proof.
Read the guidesaas
AI crawler robots.txt policy for SaaS websites
A SaaS-friendly robots.txt pattern that keeps marketing and docs crawlable while protecting app and customer areas.
Read the guideecommerce
AI crawler policy for ecommerce product and collection pages
How ecommerce sites can keep product discovery open while avoiding cart, checkout, account, and search-result traps.
Read the guidelocal seo
AI search pages for local businesses: what to publish first
A practical content order for local businesses that want search and AI assistants to understand services, location, and proof.
Read the guideconsultants
AI visibility audit checklist for consultants
A client-safe checklist consultants can use to review crawlability, llms.txt, robots.txt, internal links, and proof.
Read the guidehelp centers
How to map a help center into llms.txt
A simple way to choose help center categories, docs, status pages, and escalation links for an llms.txt file.
Read the guidellms.txt for a knowledge base
A knowledge-base-specific template for top categories, troubleshooting paths, policies, status, and escalation pages.
Read the guidedocs
AI crawler readiness for documentation sites
A documentation-site checklist for public docs, canonical URLs, changelogs, API references, and crawler proof.
Read the guidellms.txt for API docs
How API documentation teams can use llms.txt to point crawlers toward quickstarts, references, SDKs, changelogs, and examples.
Read the guideblog seo
AI search visibility checklist for blogs
How blogs can organize evergreen posts, sources, internal links, and llms.txt entries for AI-era discovery.
Read the guidewordpress
WordPress llms.txt setup checklist
A WordPress-focused checklist for publishing llms.txt, keeping robots.txt clean, and linking important public pages.
Read the guideshopify
Shopify llms.txt setup for stores
A Shopify-oriented guide for deciding what to include in llms.txt and how to avoid cart and checkout traps.
Read the guidewebflow
Webflow llms.txt publishing options
How Webflow teams can publish a root llms.txt file using built-in hosting, reverse proxy, or edge rules.
Read the guidewix
Wix llms.txt checklist for small sites
A Wix-friendly checklist for deciding whether to use native settings, redirects, or external hosting for llms.txt.
Read the guideframer
Framer llms.txt for marketing sites
A Framer-focused llms.txt checklist for marketing pages, CMS collections, docs, and case studies.
Read the guideexamples
Copy-ready llms.txt example for SaaS companies
A practical SaaS llms.txt structure for product, docs, pricing, security, changelog, and support pages.
Read the guideCopy-ready llms.txt example for ecommerce stores
A store-focused llms.txt template for collections, buying guides, product policies, and support pages.
Read the guideCopy-ready llms.txt example for local service businesses
A local-service llms.txt template for services, locations, proof, FAQs, policies, and contact routes.
Read the guideCopy-ready llms.txt example for documentation sites
A docs-focused llms.txt template for quickstarts, API references, changelogs, SDKs, and status pages.
Read the guideworkflow
Sitemap to llms.txt workflow for large sites
How to turn a sitemap into a concise llms.txt draft without dumping every URL into the file.
Read the guidequality control
Validate llms.txt before publishing: a safe review checklist
A pre-publish review process for llms.txt structure, links, private-data risk, and maintenance notes.
Read the guidelogs
Apache log proof for AI crawler visits
How to read Apache access logs for Googlebot, OAI-SearchBot, GPTBot, PerplexityBot, Applebot, and CCBot visits.
Read the guidenginx log proof for AI crawler visits
A practical nginx access-log checklist for proving crawler access to robots.txt, sitemap.xml, llms.txt, and key pages.
Read the guideCloudflare logs for AI crawler troubleshooting
How to use Cloudflare, WAF, and origin logs together when AI or search crawlers cannot fetch important pages.
Read the guideGooglebot and OAI-SearchBot log proof: what to measure
A narrow checklist for proving Googlebot and OAI-SearchBot reached the files that matter.
Read the guideindexnow
IndexNow playbook for a new site
How to use IndexNow as a discovery notification without pretending it guarantees indexing or rankings.
Read the guidesearch console
Search Console first-week routine for a new domain
A practical first-week Search Console routine after verification, sitemap submission, and priority URL inspection.
Read the guidedistribution
UTM distribution without spam
How to use UTM links for useful community distribution without fake clicks, link drops, or manipulative search behavior.
Read the guidecommunity
Reddit answer-first traffic checklist
A white-hat way to answer relevant Reddit threads without turning the site into a spam campaign.
Read the guidegithub
GitHub README snippet for AI crawler policy
A README-friendly snippet pattern for documenting llms.txt, robots.txt, crawler policy, and evidence links.
Read the guidedirectories
AI tool directory submission SEO checklist
How to submit a tool to directories with honest descriptions, canonical URLs, and proof tracking.
Read the guideAI tool directory description template
A reusable description template for AI tool directories that avoids duplicate fluff and unsupported claims.
Read the guidelaunch
Product Hunt launch prep for SEO and AI visibility tools
A practical preparation checklist for launching a free SEO or AI visibility tool without overclaiming results.
Read the guideShow HN checklist for an AI SEO tool
A Show HN preparation checklist for presenting a free AI crawler or llms.txt tool honestly.
Read the guidelink policy
nofollow, ugc, and sponsored links for community promotion
How to think about nofollow, ugc, and sponsored attributes when measuring free traffic and links.
Read the guidetraffic
Legal free traffic tactics that can compound
A practical shortlist of free traffic tactics that are legal, useful, and measurable without fake clicks.
Read the guideWhy not to use Google search URL tricks for SEO
Why posting Google search URLs with your brand query is a weak tactic compared with direct useful links and real content.
Read the guidecontent quality
Content quality checklist for AI search visibility
A people-first content checklist for AI-era search pages that need to be useful, sourced, and worth linking.
Read the guideContent refresh without fake freshness
How to update AI crawler and SEO pages only when the content materially improves.
Read the guidesite architecture
Internal linking plan for a new SEO site
How to link tools, guides, templates, and proof pages so crawlers and users can understand a new topical site.
Read the guidestructured data
Schema markup for free SEO and crawler tools
A safe structured data checklist for free web tools such as validators, generators, and analyzers.
Read the guideFAQ schema without spam for AI visibility pages
How to use FAQ schema only when it reflects visible, helpful questions on the page.
Read the guidefeeds
RSS, Atom, and JSON Feed for discovery
Why discovery feeds still matter for crawlable updates, directories, automation, and proof.
Read the guideagencies
llms.txt for agencies: client-safe setup
A client-safe llms.txt workflow agencies can use without exposing private strategy or client data.
Read the guidebadges
AI crawler policy badge use cases
When a crawler policy badge can help trust and referral traffic, and when it is just decoration.
Read the guidemaintenance
Monthly AI crawler audit checklist
A monthly maintenance checklist for robots.txt, llms.txt, sitemap, feeds, logs, and crawler documentation changes.
Read the guideAI crawler policy changelog template
A changelog template for documenting crawler policy changes, source checks, and proof after each update.
Read the guidepolicy
Training opt-out policy page checklist
How to create a clear public policy page for AI training-use preferences without confusing it with search crawling.
Read the guidetechnical seo
robots.txt, noindex, and canonical: the difference
A plain-English explanation of three controls that are often confused during SEO and AI crawler setup.
Read the guideCrawl budget for small sites: what actually matters
A small-site checklist for helping crawlers spend time on canonical pages instead of duplicate or blocked paths.
Read the guideprivacy
Privacy-safe llms.txt checklist
How to keep llms.txt useful without exposing private, customer-specific, or unpublished information.
Read the guideAI crawler policy for private docs and staging sites
How to protect private documentation and staging sites without relying on robots.txt as security.
Read the guideai search
AI Overviews: why standard SEO still matters
Why crawlability, helpful content, internal links, and structured pages still matter when AI search features appear.
Read the guidemeasurement
Search Console performance export: what to save
What to export from Search Console when a new site starts getting impressions, clicks, and query data.
Read the guideServer logs vs analytics: which proof to use
How to separate crawler proof, human sessions, referrals, and organic performance evidence.
Read the guidepublishers
AI crawler policy for publishers
A publisher-focused checklist for balancing search visibility, content licensing, training-use controls, and crawl proof.
Read the guidecontent operations
AI search content brief template
A content brief template for creating pages that answer one specific job with sources, examples, and internal links.
Read the guideDaily blogging without thin content
How to publish daily during a launch sprint while keeping every page useful, sourced, and internally linked.
Read the guideFirst 50 posts for an AI crawler visibility site
A topical map for the first 50 posts on a site about llms.txt, AI crawlers, Search Console, logs, and legal traffic.
Read the guideproof
AI SEO proof dashboard: what to show
What an AI SEO proof dashboard should show so stakeholders can separate setup, crawl proof, distribution, and organic results.
Read the guide