Industry5d ago
Cloudflare CEO Predicts Pay-To-Crawl Web
Bot traffic now exceeds human traffic on the internet. Matthew Prince attributes this surge to the proliferation of AI agents, arriving years earlier than his 2027 forecast. He predicts a transition to a pay-to-crawl model for web access. This shift forces Cloudflare and other providers to monetize the automated scraping of site data.