Is Your Website AI Search Ready?

Traditional search has a predictable loop: Google crawls your site, indexes it, ranks it against competitors, and returns a list of blue links. A user picks one, visits, reads, and forms an opinion. Your site gets traffic, analytics show the visit, and you can measure the impact.

AI search breaks that loop. When someone asks ChatGPT Search, Perplexity, Google AI Overviews, or Claude.ai a question, the agent fetches content from multiple sources, synthesizes a single answer, and presents it directly. The user may never see a link to your site. They get the answer, not the link.

This is not a future concern. These tools are running right now, and search behavior is shifting toward them. The implication for your website is straightforward: if an AI agent cannot parse, understand, and confidently cite your site, you are not in the answer. You are not even in the conversation.

“The shift from a human-read web to a machine-read web is the biggest architectural change in content discovery since the first search crawler.”

This shift is not about abandoning traditional SEO. It is about recognizing that your content now needs to serve two audiences at once: the human reader who clicks through, and the AI agent that may never return a click at all.

The gap between what humans experience on your site and what an AI agent can actually read is often wider than website owners realize. Agents do not execute JavaScript, they do not see hover states or modal windows, and they do not follow interactive navigation. They work with what the server sends.

The signals agents can actually read include: the raw text inside your HTML, JSON-LD structured data blocks embedded in your page head, meta description and title tags, the instructions in your robots.txt file, the pages listed in your sitemap.xml, and, if you have created one, the contents of your llms.txt file.

Common problems that make sites partially or fully invisible to agents: content that only appears after JavaScript hydration, text inside image carousels or tabs that are not server-rendered, no structured data markup, a missing or outdated sitemap, and no clear organizational identity baked into the page markup. These are not edge cases. They describe the majority of business websites built over the last decade.

“If your site requires a browser to make sense of it, it requires a human. Agents are not human.”

The good news is that none of these problems require a full rebuild to fix. Most can be addressed at the content and configuration layer, without changing how the site looks or works for the humans who visit it.

The AgentReady open standard (launched in March 2026 and maintained by ora.run) defines a concrete, versioned specification for what it means for a website or product to be usable by AI agents. It organizes requirements into five categories, each addressing a different part of the agent interaction.

1. Discoverability

Can agents find your content at all? This pillar covers the files and signals that tell agents what your site contains and how to crawl it: a robots.txt file with explicit AI crawling policy, a valid sitemap.xml, and an llms.txt file that provides a structured reading guide for AI. This is the entry point for agent readiness, and it is where most sites should start.

2. Content for Agents

Once agents reach your pages, can they extract meaning without running JavaScript? This pillar covers JSON-LD structured data (schema.org markup for your organization, services, and content), semantic HTML that clearly signals heading hierarchy and article structure, and agent-readable markup that does not require a browser to interpret.

3. Capabilities

Can agents understand what your product or service does and how to interact with it programmatically? If you have a public API, this means publishing an OpenAPI 3.1 spec. If you expose tools for agents to call, it means implementing the Model Context Protocol (MCP). For most marketing and brand sites, this pillar is not yet applicable, but for SaaS products it is already table stakes.

4. Identity and Access

For gated or authenticated content, this pillar asks whether you use OAuth 2.0 correctly so agents can act on behalf of authenticated users. For most marketing and informational sites this is not relevant today, but it becomes critical for any site with user accounts and protected content.

5. Commerce

The emerging frontier: can agents complete transactions on behalf of users? Protocols like x402 and the Agentic Commerce Protocol (ACP) are still early, but they define how agents will eventually purchase, subscribe, and transact. This is an area to watch, not necessarily to implement today, unless you are building specifically for agentic commerce.

The practical takeaway: most business websites need to focus on pillars 1 and 2 right now. Pillars 3 through 5 are more relevant for API-driven products, SaaS platforms, and transactional sites.

Agent readiness does not require a full site redesign. The four highest-impact changes are configuration and content additions that sit on top of your existing site.

Priority 1: robots.txt with an explicit AI policy

Right now, 96% of websites are silent on the topic of AI crawling. Agents treat silence as permission today, but that assumption is not guaranteed to hold as standards evolve. Add explicit directives for the AI crawlers that matter: GPTBot (ChatGPT), ClaudeBot (Anthropic), PerplexityBot, and Googlebot-Extended. If you want to be crawled and cited, Allow: /. If you have content you want to keep private, set it now before it gets indexed.

Priority 2: llms.txt

A plain-text file at yourdomain.com/llms.txt gives AI agents a structured reading guide: what your site is, what is on it, where the important pages are, and what each one covers. Think of it as a curated table of contents written specifically for machines. Early adopters get indexed and cited first, for the same reason early adopters of sitemap.xml got better search coverage in 2005.

Priority 3: JSON-LD structured data

Structured data is what AI cites when it talks about your organization. At minimum, add an Organization schema block to your homepage with your name, description, URL, contact details, and social profiles. Add FAQPage schema to any FAQ or service pages. Add BreadcrumbList to interior pages. These blocks are machine-readable without JavaScript and are how agents build a confident picture of who you are and what you do.

Priority 4: a valid, current sitemap.xml

Only 78% of sites have a sitemap at all. If yours is missing, generate one and link to it from robots.txt. If it exists, check that it is up to date and that all URLs are live. A stale sitemap with dead links signals to agents that the site is not actively maintained, which affects how confidently they will cite it.

The fastest way to know where you stand is to run your site through one of the free public scoring tools. You do not need to interpret the spec yourself: the tools report per-requirement results with specific, actionable feedback.

• ora.run Deep Scan is the official implementation of the AgentReady spec. It evaluates every applicable requirement (each identified by a stable ID like AR-DISC-01 or AR-CONT-01) and tells you exactly what passes, what fails, and what is not applicable for your type of site.
• Cloudflare Agent Readiness scores across four dimensions: Discoverability, Content, Bot Access Control, and Capabilities. It provides actionable feedback modeled on the Lighthouse audit experience, making it approachable for teams already familiar with performance scoring.

A realistic baseline for most sites today: they will pass the robots.txt presence check (78% do) but fail on AI-specific policy, structured data, llms.txt, and everything in the Capabilities pillar. Passing the Discoverability basics, which takes a few hours of work, already puts you ahead of 96% of the web on the metric that matters most to agents right now.

The performance case for early action is concrete. Cloudflare measured that sites optimized for agent consumption required 31% fewer tokens for AI to process and answered questions 66% faster than unoptimized sites. Agents surface content they can process efficiently and confidently. Sites that are hard to parse get deprioritized in synthesis, even if their content is the most relevant.

“Only 4% of websites declare AI preferences. The early-mover window is open right now.”