Cloudflare Splits AI Bot Controls
Cloudflare now lets customers manage AI crawlers by Search, Agent, and Training behavior, with new defaults scheduled for September 15.
The News
Cloudflare documentation says all customers can manage AI crawler traffic by behavior: Search, Agent, and Training. New domains will receive updated defaults on September 15, 2026, blocking Training and Agent bots on ad-supported pages while keeping Search allowed.
The OPTYX Analysis
Crawler policy is moving from one blunt AI-blocking switch to intent-based access control. Search visibility, agent retrieval, and model training are being separated into different commercial and operational categories.
Enterprise AI Impact
Publishers and brands should audit Cloudflare settings before the September change. Misclassification or inherited defaults could affect answer-engine inclusion, AI agent access, training restrictions, referral measurement, and monetization strategy across ad-supported content.