How Cloudflare Is Adapting the Web for AI Agents

Cloudflare’s 16th Birthday Week announcements address a web where AI agents and crawlers are a growing audience alongside people. Cloudflare says automated traffic passed human traffic on its network in May 2026, earlier than its previous forecast of the second half of 2027. It also projects that automated traffic could reach 1,000 times human traffic within five years if current trends continue—a forecast, not a measurement of today’s web.31

The product response has three parts: ways for publishers and businesses to charge for machine access, more precise choices about how AI systems may use content, and tools for identifying security threats. The announcements do not provide complete pricing or eligibility details for every product.

Payments: charging for access and content use

Monetization Gateway lets domain owners charge AI agents for access to resources including websites, APIs, MCP tools and datasets, using an HTTP 402 payment flow. It is in a closed beta; U.S.-based sellers can apply. Cloudflare says customers are already using it to charge for tokens, APIs and MCP tools, but the available announcement does not specify general pricing or broader eligibility terms.18

Pay Per Use addresses a different transaction: AI companies report when they use publishers’ content, while Cloudflare handles billing, payouts and reporting so publishers can be paid according to use. Cloudflare describes Pay Per Use as being in beta. The available information does not establish payout rates or complete participation requirements.13

In short, Monetization Gateway is for charging an agent before it accesses a resource; Pay Per Use is intended to compensate publishers when participating AI companies report using their content. Neither announcement, in the provided details, supplies a full price schedule.

Content controls: search does not have to mean training permission

Cloudflare’s Disallow AI Training setting lets a website refuse AI training use while remaining discoverable in search. The company says it is replacing a single “Block AI Bots” switch with separate controls for search, AI training and AI agents. That gives site owners more specific choices than allowing or blocking all AI-related access together.26

Security: threat intelligence and application behavior

Cloudflare says it is expanding access to its Threat Events Platform to every Cloudflare account. Its Threat Signals tool processes selected open-source threat reporting, extracts and normalizes indicators, and connects threat context to WAF rules. Cloudflare says Threat Signals is free for every account.7

Application Profiles takes a different approach: it learns the expected structure and format of an application’s HTTP requests and identifies deviations. The aim is to help teams enforce a positive security policy based on what valid requests should look like. Cloudflare announced the product on September 29, 2026; the available product description does not specify its price or plan eligibility.20

What the traffic figures do—and do not—show

The May 2026 crossover is Cloudflare’s report about traffic on its own network, not evidence that automated traffic has already reached 1,000 times human traffic across the entire internet. That larger figure is a conditional five-year forecast. Taken together, the figures explain the business problem behind the rollout: more machine access can bring costs and security risks, while not necessarily producing the human visits that support advertising. Cloudflare’s announced tools aim to give site owners ways to set access terms, distinguish search from training, and defend applications as automated use grows.1731