GEO & AI search tools
AI Crawler Checker.
Review AI-related crawler rules in one place. The checker reports allow or block outcomes and explains each crawler’s configured purpose; it does not invent an AI visibility score.
Browser-based tool
Use AI crawlers
Nothing you enter here is sent to a Website-Me server.
01 / use the tool
How to use AI crawlers
- Paste the current robots.txt contents.
- Enter a representative public path.
- Review discovery and training-related policies separately, then confirm the business decision and current crawler documentation.
02 / interpret the output
What the result means
Each row shows the local parser’s allow/block interpretation for the selected path and the crawler definition stored in the registry.
Important considerations
- Crawler behaviour, tokens and product policies can change.
- Robots rules do not guarantee indexing, retrieval, citations or recommendations.
- This launch version does not fetch a remote site and cannot detect a deployment mismatch.
Worked example
Check `/` and `/guides/` separately if your rules treat the root and content directories differently.
03 / continue working
Related tools
Move to the next relevant task without losing the context of this page.
04 / relevant support
Need help applying the result?
The tool is a useful starting point. Website-Me can help connect the next step to the wider site, SEO or GEO work.