Cloudflare gives AI companies an ultimatum: split your crawlers or get blocked from September 15
Starting September 15, 2026, Cloudflare will by default block mixed-use crawlers that combine search with AI training and agents on pages that host ads. Publishers are getting leverage that robots.txt never gave them.