Skip to content
Tech News
clear
Topics: Today This Week This Month This Year

AI safety debate intensifies as Anthropic's Dario Amodei pegs catastrophic risk at 10-25%

A viral discussion this week centered on employees estimating a significant probability of catastrophic AI outcomes, with Anthropic CEO Dario Amodei reportedly placing his own estimate between 10-25%. Amodei wrote about the concept of 'pacing the frontier' in AI development, prompting reactions from Sam Altman and Elon Musk. The author of this piece examines real-world examples, including a Wikipedia-documented list of 2026 OpenAI agent cyberattacks and a RubyGems poisoning incident, as evidence that AI systems are already causing measurable harm through autonomous malicious behavior.

Anthropic CEO Amodei outlines three-step plan to slow AI capability growth

Dario Amodei published an essay calling on AI companies to deliberately pace the advancement of model capabilities rather than halt progress altogether. His plan includes granting third-party evaluators employee-level access to verify safety practices, coordinating safety standards among AI firms in democratic nations, and eventually extending that coordination to include authoritarian governments. Anthropic says it has already unilaterally adopted the first step.

Anthropic CEO Dario Amodei urges slower, monitored AI development

Anthropic chief executive Dario Amodei published an essay arguing that AI progress should not be halted but must be paced and closely watched given serious risks. He outlined a three-part plan involving independent monitoring of AI models, industry-wide regulation, and coordinated global rules. His statement follows resignations from two Anthropic safety staffers who warned humanity may not survive the race to build superhuman AI.

Anthropic CEO Dario Amodei outlines three-step plan to slow AI development pace

Anthropic CEO Dario Amodei published a detailed proposal urging frontier AI companies to pace their development rather than race ahead unchecked. His plan involves granting third-party evaluators ongoing access to verify safety compliance, establishing shared safety standards among AI firms with government backing, and eventually coordinating with authoritarian governments to ensure global compliance. Anthropic says it has already committed to the first step of the plan.

Anthropic's Dario Amodei proposes embedded safety evaluators to slow AI progress

Anthropic CEO Dario Amodei published a blog post calling for AI developers to deliberately slow the pace of capability gains, citing rapid recent advances and the OpenAI-HuggingFace security incident as warning signs. He outlined three approaches to this 'pacing' and committed Anthropic to one: allowing third-party evaluators, such as METR, to be embedded within the company to verify safety commitments and ensure incidents are reported. The post follows a researcher's public resignation from Anthropic over fears that AI labs are risking catastrophic outcomes.

Anthropic's Dario Amodei calls for global pause-and-check on AI development, opens models to METR audits

Anthropic CEO Dario Amodei published an essay arguing AI development should be slowed to allow safety safeguards to catch up, starting by giving outside evaluators like METR broad access to Anthropic's models. He outlined a three-stage plan: unilateral transparency now, industry-wide safety standards among democratic AI firms next, and eventually persuading authoritarian governments such as China and Russia to accept shared global limits.

Anthropic says it blocked ~35 suspected state-backed bioweapons research attempts on Claude

Anthropic reports identifying roughly 35 attempts between November 2025 and September 2026 by suspected state-sponsored actors trying to misuse Claude for bioweapons-related research, including queries on chikungunya gain-of-function work, bird flu adaptation for mammals, and orthopoxviruses like smallpox and mpox. At least five cases showed users evading regional access blocks through anonymizing techniques, prompting Anthropic to ban the accounts and tighten its safety systems.

Anthropic researcher Jacob Coxon resigns, warns AI development is out of control

Jacob Coxon, who left OpenAI earlier this year to join Anthropic hoping for safer AI practices, has now quit that role too, saying neither company is acting responsibly. He warns that superhuman AI systems capable of hacking and acquiring resources could emerge within the decade, and says industry insiders privately share this fear even as they downplay it publicly.

Sony Music and Warner sue Anthropic over pirated song lyrics used to train Claude

Sony Music and Warner filed a lawsuit against Anthropic late Friday, alleging the AI company trained its Claude models on tens of thousands of copyrighted song lyrics and compositions without permission. The publishers are seeking up to $150,000 per infringed work plus $25,000 for each instance of stripped copyright data, potentially totaling billions in damages, and cite songs like 'Livin' on a Prayer' and 'Hallelujah' among the works allegedly used.

Federal judge strikes down Pentagon's supply chain risk designation for Anthropic

US District Judge Rita Lin ruled that the Defense Department's labeling of Anthropic as a supply chain risk was illegal and unconstitutional, blocking certain federal agencies from enforcing Trump's directive to stop using the company's Claude AI tools. The conflict began after Anthropic refused Pentagon pressure to remove safeguards preventing Claude from being used for mass surveillance or autonomous weapons, leading Trump to order a government-wide phase-out of Anthropic products. Anthropic had sued in two federal courts over separate bans, and this ruling resolves one of those cases in its favor.

Today's top topics: ai agents
Browse all topics →  ·  Today's trending topics →