Cloudflare just segmented AI traffic into three types: Search, Agents, and Training, and they’re letting their customers turn access on or off for each type. That’s better than a single on/off switch for bots, but for a high-end website that isn’t enough control.
The real control businesses need is for each service within those categories. They need to decide which crawler gets which content, for what purpose, and under what terms.
This isn’t hypothetical. One of WordPress VIP’s enterprise customers might want to allow every bot to access their site except Perplexity, because they’re negotiating a deal with them. Another maybe has a deal with Perplexity and loves them, but needs to block OpenAI. And many customers just want all bots redirected to TollBit to collect micropayments for accessing their content.
Giving three on/off switches isn’t enough. Enterprise customers need fine-grained controls, down to the individual service.
Brian Alvey 