Solution
Make your site readable to AI crawlers
Most AI visibility problems are technical before they're editorial. If GPTBot can't fetch the page, nothing else you do matters — and a surprising number of sites block it without knowing.
The technical layer of AEO is concrete work with binary answers: are AI crawlers allowed, is content server-rendered, is schema valid, does llms.txt exist and parse. It's the highest-leverage place to start because the fixes are unambiguous and you can verify them the same day.
What makes this hard
The specific problems this creates — not generic advice about “the AI era”.
AI crawlers are blocked by default in many setups
GPTBot, ClaudeBot, PerplexityBot and Google-Extended are separate agents. A robots.txt written for Googlebot, or a CDN bot rule, often blocks them silently.
Client-side rendering hides your content
Several AI crawlers don't execute JavaScript. A React app that renders content client-side can be effectively empty to them.
llms.txt is new and easy to get wrong
The format is still stabilising. An invalid or missing file means assistants fall back to guessing your site structure.
The questions your buyers are actually asking
Developer-intent prompts are usually answered from documentation. If your docs aren't crawlable by AI agents, you're invisible at exactly the moment an engineer is choosing a tool.
- “How do I integrate [your product] with [common stack]?”
- “Does [your product] have an API? What can it do?”
- “What's the best [category] tool with good developer docs?”
- “How do I migrate from [competitor] to [your product]?”
- “Is [your product] open source or self-hostable?”
Replace the bracketed terms with your own. Every one of these returns a different answer depending on which assistant you ask.
What to measure
The metrics that matter for this work, and why each one earns its place on a dashboard.
Crawler access
Per-agent verification that GPTBot, ClaudeBot, PerplexityBot and Google-Extended can reach your pages.
llms.txt validity
Whether the file exists, parses, and describes your real site structure.
Schema coverage
Valid JSON-LD gives assistants unambiguous entity facts instead of inference.
Render mode
Whether critical content is present in server-rendered HTML.
How AEOVisor helps
The parts of the product that do the work described above.
Crawl simulation per agent
Simulate GPTBot, ClaudeBot, PerplexityBot and Google-Extended against your robots.txt and see exactly which paths each one can and can't reach.
llms.txt generation and validation
Generate a compliant file from your sitemap, and validate an existing one. Free, no signup.
Public API
Pull visibility, citation and audit data into your own dashboards or CI. Available on the Agency tier with API key auth.
Developer teams: common questions
Practical answers, including where this is genuinely hard
At minimum GPTBot and OAI-SearchBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot, and Google-Extended for Gemini and AI Overviews grounding. Allowing them is what makes your content eligible to be cited. If you have a reason to block training crawlers specifically, note that search-time agents are usually separate user agents.
Other teams using AEOVisor
SEO & AEO teams
You already own organic performance. Now leadership wants to know what happens when ChatGPT answers the question instead of Google — and whether the traffic you're losing is going somewhere you can influence.
Read moreContent teams
Being told you're invisible in AI search isn't useful on its own. What you need is the specific prompt you're losing, the competitor being cited instead, and the page that would fix it.
Read morePR & brand teams
Millions of buying conversations now start with an assistant. What it says about you when nobody's watching is your brand narrative — and unlike a press release, you don't get to write it.
Read more