· By Martín Garay
SEO for AI (GEO) Technical SEO Cases and lessons learned

What we learned from reviewing our site for AI agents

In September 2026 we ran Cloudflare's public Agent Readiness criteria against cookingmetrics.com — the same ones isitagentready.com evaluates. The result: it meets one of the four dimensions, and only halfway.

We are publishing the full measurement because this is exactly what we sell. A consultant who does not measure their own site is not credible.

What Agent Readiness measures

Cloudflare evaluates four dimensions: discovery (robots.txt, sitemap, Link headers), content accessibility (markdown negotiation, llms.txt), bot access control (Content Signals, AI crawler rules, Web Bot Auth) and capabilities (API catalog, MCP server card, OAuth discovery).

This is not classic SEO. An agent answering a question needs to find, read and trust — and all three are declared differently from how you talk to Googlebot.

What we found

Every line was verified with an HTTP request on 4 September 2026:

  • No sitemap exists at all. /sitemap.xml, /sitemap_index.xml and /wp-sitemap.xml all return 404. And robots.txt announces the second one.
  • robots.txt has two User-agent: * blocks. By specification most crawlers use only the first: the second is ignored entirely.
  • llms.txt returns 404. The file had been written and versioned in the repository for weeks and was never deployed.
  • Zero Content Signals and zero AI crawler rules. No GPTBot, no ClaudeBot, no Google-Extended.
  • cf-cache-status: DYNAMIC on the home page. Cloudflare was caching nothing: every request, human or bot, hit the origin.
  • LCP of 6.15 s on desktop and 6.20 s on mobile, field data. The budget is 2.5 s.

Why this matters more than it looks

A broken sitemap is not a configuration detail. It is the first signal a crawler reads about how reliable the site's information is: if the file you yourself announce does not exist, why trust the rest?

And there is an uncomfortable asymmetry: crawling has a budget. cf-cache-status: DYNAMIC means every bot visit pays the full render. A slow site gets crawled less, and what is not crawled does not get cited.

The llms.txt case is the most instructive of all, because it was not a technical error: the file was written, reviewed and versioned. Nobody deployed it. The gap between "solved in the repository" and "solved in production" is where most of the measurement problems we audit for clients actually live.

What we did with it

We turned it into a work order, prioritised by impact over effort rather than by dimension:

  1. Check whether AI Crawl Control is blocking AI crawlers in the dashboard. If it blocks, nothing else matters — and it is invisible from outside.
  2. Fix the sitemap. It is the cheapest fix and the one emitting the strongest negative signal.
  3. Merge the two robots.txt blocks and add explicit AI rules.
  4. Deploy llms.txt, which was already written.
  5. Apply the cache rules, which resolve the DYNAMIC and part of the LCP at once.

Points 2 to 5 are hours of work and move three of the four dimensions.

What this article does not solve

Agent Readiness measures whether your site is readable by an agent. It does not measure whether what it says is worth citing. A site with a flawless sitemap and generic content will be found and discarded.

We have not measured the outcome yet either. We are publishing the dated baseline; the second measurement comes after applying the five points, and we will publish it whatever it says.

If you want us to run this same measurement on your site, start here.

Frequently asked questions

Does Agent Readiness replace technical SEO?

No. It extends it. Sitemap, robots.txt and speed are classic SEO; Content Signals, llms.txt and MCP cards are the new layer. A site can rank well on Google and be invisible to an agent.

Is it useful if my business does not sell software?

Yes. The discovery and accessibility dimensions apply to any site. The capabilities one — API catalog, MCP — only applies if you have something an agent can execute.

Does declaring ai-input=yes mean giving the content away?

They are independent permissions. ai-input authorises AI to use the content to answer and cite you; ai-train authorises training models on it. They can be declared separately, and most businesses that want traffic want the first and not the second. OpenAI and Perplexity draw the same line with their own bots: we take it apart, with the official quotes, in what GEO is.

How often should this be measured?

At least after every infrastructure change. Our own case proves it: llms.txt was written and the deployment left it out without any alert noticing.