What we learned from reviewing our site for AI agents
A review of our site using AI agent access criteria: what we observed, how we verified it and what still needs investigation.
BLOG
Technical SEO
Sitemaps, robots.txt, Cloudflare and crawling: the configuration that decides whether Google, Bing and AI bots can read your site, explained with steps and evidence.
What GEO is: how to create content for generative search
What Generative Engine Optimization means, what the study that coined the term measured and what Google says. With official OpenAI and Perplexity sources.
Content Signals in robots.txt: a use declaration, not a block
The robots.txt directive Cloudflare created in 2025: how to write it, what search, ai-input and ai-train mean, and why it declares use without blocking any access.
Robots.txt editor: one switch per bot, published via Cloudflare
Your platform writes robots.txt and Cloudflare rewrites it. How the editor works: it reads your real file, one switch per bot, and publishes the result through a Worker.
Bot accessibility testing: three layers, two questions
How to verify crawlers can read your catalog and are blocked from cart and checkout: robots.txt, firewall and server fail differently, so all three layers need testing.
Cloudflare for SEO and GEO: technical configuration guide
DNS, TLS, cache, Bot Fight Mode and AI Crawl Control: the exact Cloudflare settings that decide whether Googlebot, Bingbot and AI search bots can see your store.
XML sitemaps: verified best practices and the myths to bury
What actually matters in your sitemap: honest lastmod, the changefreq and priority myths, Bing's daily re-reads, and why freshness now feeds AI-generated answers.