AI bots scraping your data? This free tool gives those pesky crawlers the run-around
Published on: 2025-06-02 17:56:59
Eugene Mymrin/Getty Images
The rise of AI-generated content, also known as synthetic media, has mostly caused problems: It helps spread misinformation, steal from artists, and erode trust in what we see online. However, Cloudflare may have found a use case where artificial intelligence could help protect original content from the tentacles of AI companies.
On Wednesday, the company released AI Labyrinth, a tool that uses AI-generated content to "slow down, confuse, and waste the resources" of unauthorized AI crawlers.
Also: Chatbots are distorting news - even for paid users
Multiple studies have found that AI chatbots -- including ChatGPT and Perplexity -- are still accessing content from sites that block their crawlers. Cloudflare noted in the announcement that crawlers "generate more than 50 billion requests to the Cloudflare network every day or just under 1% of all web requests we see" -- and how you block them matters.
"While Cloudflare has several tools for identifying and b
... Read full article.