Cloudflare's September 15 AI Bot Change: What to Check Before It Lands

If your site sits behind Cloudflare — directly, through your host, or through a security plugin that quietly uses it — a setting is changing on September 15, 2026 that's worth five minutes of attention beforehand. Not because the change itself is bad, but because the way it's built creates a genuine, documented way to block Google along with the AI crawlers you were actually trying to stop.

What's actually changing

Cloudflare is retiring its single “block AI bots” switch and replacing it with three separate categories:

• Search — crawlers that index your site to answer questions about it later. This is Googlebot, Bingbot, and the search-specific AI crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot).
• Agent — bots acting on a person's behalf in real time, fetching a specific page because someone asked an AI assistant about it right now (ChatGPT-User is the common example).
• Training — crawlers pulling your content in to train or fine-tune a model, permanently, with no live citation attached (GPTBot, ClaudeBot, CCBot).

Starting September 15, any new site connecting to Cloudflare gets Training and Agent blocked by default on pages that run ads, while Search stays allowed. If you're already a Cloudflare customer, nothing switches under you automatically — Cloudflare is prompting existing accounts to make an explicit choice before that date, not changing it for you.

The part worth being careful about

Cloudflare says this directly in its own announcement: Googlebot, Bingbot, and Applebot are explicitly multi-purpose crawlers — the same bot handles both search indexing and AI training crawling. Cloudflare's rule applies the most restrictive setting across everything a bot does. So choosing to block Training doesn't leave Googlebot's search behavior untouched — it blocks Googlebot outright, indexing included. There is no configuration that keeps the search half of a multi-purpose crawler while blocking the training half. If you set out to protect your content from AI training and accidentally tank your Google rankings instead, this rule is why.

Do you actually need to do anything?

Only if your site has its own Cloudflare account — not every site behind Cloudflare does. Squarespace, Showit, and Pixieset manage their own CDN and security layer invisibly; there's no separate dashboard for you to log into, so this deadline is a non-event on those platforms. Self-hosted WordPress is the common case where it matters: many hosts and security plugins (Wordfence and others) route traffic through Cloudflare's free tier, and plenty of site owners have a Cloudflare login they've never had a reason to open.

If you're not sure which situation you're in, ask your host directly: “Do I have my own Cloudflare account for this domain, separate from whatever caching you manage?” If the answer is yes, log in before September 15 and check Security → Bots. Whatever you decide about Training and Agent, leave Search allowed — that's the category your actual rankings run on. This mainly bites sites running ads or affiliate content, since that's the specific case the new default targets.

A separate lever: robots.txt

Blocking training crawlers doesn't require touching your CDN at all — a robots.txt rule works too, and it's available on every platform, not just self-hosted ones. The distinction that matters: robots.txt is a request a crawler chooses to honor, not enforcement. Most named training crawlers (GPTBot, ClaudeBot, CCBot) do respect it — the one notable exception reported is Bytespider, which is documented ignoring it. So robots.txt stops the well-behaved crawlers and does nothing about the ones that don't care, while a CDN-level block is real enforcement but carries the multi-purpose-bot risk above. They're not a choice between one or the other — most sites end up wanting both, understood as separate tools with separate failure modes.

Whichever lever you use, keep the same principle from Google's AI opt-out setting: a rule aimed at “keep my content out of AI training” is a different decision from “stop showing up when someone asks an AI who's good at what I do” — and it's easy to write a rule that accomplishes the first while accidentally causing the second.