In September 2026, Cloudflare, Inc. launched new AI crawling controls, including a “Disallow AI Training” setting and an Accountable designation for AI crawlers, giving website owners granular control over search, AI training, and AI agents while keeping them visible in search results. A key innovation is Cloudflare’s Bot Preference Sync and accountability criteria, which standardize how websites express AI access preferences across major crawlers like Apple, Google, and Microsoft, aligning…
Source: Do Cloudflare’s (NET) New AI Crawler Controls Quietly Strengthen Its Data-Moat Narrative?