#ai crawlers

Robots.txt AI Training Opt-Out Gets One Shared Token

Robots.txt AI training opt out now has one shared token that Google, Microsoft, OpenAI and Anthropic say they will honor. What it does and what it can't.