Tech News
← Home  ·  All topics

Ai Scraping

11 GoKawiil briefs on this topic · RSS for ai scraping

Archive of Our Own restored after 17-day security outage

The Organization for Transformative Works announced on November 15 that Archive of Our Own (AO3) is back online following a security incident that took the fan fiction site offline for 17 days. Some features, including user icons and external work importing, remain disabled to speed up restoration, and the OTW warned of possible loading delays due to high traffic. The organization said it has no reason to believe the user database was compromised but could not rule it out.

New York Times sues Brave browser maker over AI scraping and paywall bypass claims

The New York Times has filed a lawsuit against Brave, maker of the Brave web browser, alleging the company's AI tools scrape its content and bypass paywalls without authorization. The suit reportedly centers on how Brave's AI features access and summarize Times articles.

NYT, Daily News, Tribune and Orlando Sentinel sue Brave over AI scraping, paywall bypass

The New York Times and a group of publishers including the New York Daily News, Chicago Tribune and Orlando Sentinel have filed lawsuits against Brave Software. The complaints allege Brave scraped their websites without permission, sold access to its content index to AI firms such as Anthropic, Perplexity, Mistral, xAI and Cohere, and built a browser feature that lets users bypass publisher paywalls.

OpenAI, Google, Microsoft and Anthropic back shared opt-out standard for AI scraping

The four AI companies say they will support a new 'policy token' signal, proposed by Anthropic's Kevin Kelley through the Internet Engineering Task Force, that lets publishers set AI training preferences once within their robots.txt file rather than contacting each company separately. Anthropic has already adopted the signal, OpenAI says it will implement it, and Google and Microsoft plan to support it next year, with Microsoft also offering separate controls for AI training versus AI grounding.

Reddit to require logins for most browsing, starting with tests outside the US

Reddit announced it will shift to a primarily logged-in experience, limiting access to content for users without accounts. CEO Steve Huffman said the homepage will remain open, and links from search engines or shared links will still work, but general browsing or visiting specific subreddits will require login. Testing begins this quarter outside the US before a wider rollout.

Reddit to push users toward logged-in access as search referrals fall

Reddit announced during its Q3 earnings call that it will shift toward a primarily logged-in experience, citing a drop of more than 1 million weekly search referrals in October. The company's global daily active users held flat at 130.3 million, but U.S. daily actives fell to 50.7 million from 53.2 million in Q2, a 2% year-over-year decline. Reddit warned daily active users could drop by a low double-digit percentage next quarter if the trend continues, even as revenue rose 53% year-over-year to $895 million.

Meta business account terms now permit AI crawling of linked websites, effective Oct. 30

Meta has updated the commercial terms for Meta Business Suite to explicitly allow its bots and automated tools to crawl, index, and use content from websites associated with business accounts, including news publishers. The change takes effect October 30 with no opt-out option; businesses must accept the terms or lose access to the platform. Meta describes the update as a consolidation of existing conditions rather than a new policy.

Reddit to restrict Old Reddit access and end RSS support by Nov 2026

Reddit announced it will limit Old Reddit's interface to users who have accessed it within the past six months, building on an earlier requirement that users log in to use the old UI. The company also said it will discontinue RSS feed support on November 13, 2026, citing large-scale scraping and automated abuse, and is pointing moderators toward Devvit apps like Discord Relay as an alternative.

Reddit will limit Old Reddit access to recent users, drop RSS support

Reddit announced it will soon restrict the classic desktop site to logged-in accounts that have used Old Reddit within the past six months, blocking new accounts and long-dormant users from the interface. Moderators are exempt from this recent-usage rule. Separately, Reddit said it will end support for RSS feeds starting November 13, citing their use for large-scale scraping.

Unsealed filings show Microsoft, OpenAI executives privately called AI training 'theft'

Newly unredacted court documents in The New York Times' copyright suit against OpenAI and Microsoft reveal that a senior Microsoft executive privately labeled the companies' AI training practices as theft, while OpenAI leaders reportedly acknowledged their models posed an existential threat to publishers. The filings also allege the companies bypassed paywalls, scraped content en masse, and removed copyright notices to build training datasets.

Anubis anti-scraper tool highlights AI industry's 'do as I say' hypocrisy on scraping

The article's page itself was blocked by Anubis, a proof-of-work tool websites deploy to deter aggressive AI scraping bots, illustrating the very problem the piece discusses. Anubis forces browsers to solve a computational puzzle before loading content, adding negligible friction for humans but real cost for mass scrapers.