Cloudflare’s new policy pushes AI companies to pay for publishers’ content

· TechCrunch AI ·

Cloudflare will block many AI crawlers by default unless companies separate search indexing from AI training and agent traffic.

Categories: Money & Moves

Excerpt

Cloudflare is giving AI companies until September 15 to separate web crawlers used for search from those used for AI training and agents, or risk being blocked by default on many publisher sites.