The CEO of the largest digital and print publisher in the U.S. has accused Google of being a bad actor for crawling its websites to support the search giant’s AI products.
Neil Vogel, CEO of People, Inc. (formerly Dotdash Meredith), a publisher that operates over 40 brands, including People, Food & Wine, Travel & Leisure, Better Homes & Gardens, Real Simple, Southern Living, AllRecipes, and others, said that Google is not playing fair because it uses the same bot to crawl websites to index them for the Google search engine as it does to support its AI features.
“Google has one crawler, which means they use the same crawler for their search, where they still send us traffic, as they do for their AI products, where they steal our content,” said Vogel, speaking at the Fortune Brainstorm Tech conference this week.
He noted that three years ago, Google Search accounted for about 65% of the company’s traffic and that has since dropped to the “high 20s.” (Vogel shared an even more startling statistic with AdExchanger last month, saying that as of several years ago, Google’s traffic accounted for as much as 90% of People Inc.’s traffic from the open web.)
“I’m not complaining. We’ve grown our audience. We’ve grown our revenue,” Vogel told conference attendees. “We’re doing great. What is not right about this is: you cannot take our content to compete with us.”
Vogel believes publishers need more leverage in the AI era, which is why he feels it’s necessary to block AI crawlers — automated programs that scan websites to train AI systems — as that can force them into content deals. His company, for example, has a deal with OpenAI, which Vogel described as a “good actor.”
People, Inc. has been leveraging web infrastructure company Cloudflare’s latest solution to block AI crawlers that don’t pay, prompting AI players to approach the publisher with potential content deals. While Vogel wouldn’t directly name the companies involved, he said they were “large LLM providers.” No deals have been signed yet, but Vogel said the company is “much further along” than before it adopted the crawler-blocking solution.
Techcrunch event Join 10k+ tech and VC leaders for growth and connections at Disrupt 2025 Netflix, Box, a16z, ElevenLabs, Wayve, Sequoia Capital, Elad Gil — just some of the 250+ heavy hitters leading 200+ sessions designed to deliver the insights that fuel startup growth and sharpen your edge. Don’t miss the 20th anniversary of TechCrunch, and a chance to learn from the top voices in tech. Grab your ticket before Sept 26 to save up to $668. Join 10k+ tech and VC leaders for growth and connections at Disrupt 2025 Netflix, Box, a16z, ElevenLabs, Wayve, Sequoia Capital, Elad Gil — just some of the 250+ heavy hitters leading 200+ sessions designed to deliver the insights that fuel startup growth and sharpen your edge. Don’t miss the 20th anniversary of TechCrunch, and a chance to learn from the top voices in tech. Grab your ticket before Sept 26 to save up to $668. San Francisco | REGISTER NOW
However, Vogel pointed out, Google’s crawler can’t be blocked because doing so would also prevent the publisher’s websites from being indexed in Google Search, cutting off that “20%-ish” of traffic that Google still delivers.
“They know this, and they’re not splitting their crawler. So they are an intentional bad actor here,” Vogel declared.
... continue reading