AI site Perplexity uses “stealth tactics” to flout no-crawl edicts, Cloudflare says
AI search engine Perplexity is using stealth bots and other tactics to evade websites’ no-crawl directives, an allegation that if true violates Internet norms that have been in place for more than three decades, network security and optimization service Cloudflare said Monday. In a blog post, Cloudflare researchers said the company received complaints from customers who had disallowed Perplexity scraping bots by implementing settings in their sites’ robots.txt files and through Web application