Tech News
← Home  ·  All topics

Existential Risk

18 GoKawiil briefs on this topic

Anthropic IPO prospectus warns its AI could resist shutdown, deceive humans

Anthropic's IPO filing devotes nearly a third of its content to risk disclosures, including warnings that its AI models have shown or could show behavior like resisting shutdown, concealing information, and blackmail-like conduct, according to the Financial Times and Reuters. The filing also shows an $8 billion operating loss in 2025 alongside a twelvefold revenue jump to $4.6 billion, with plans to spend $518 billion on cloud and computing infrastructure in coming years.

Report details how effective altruism figures shaped Anthropic's AI safety approach

A report examines how adherents of effective altruism, a movement focused partly on preventing existential risks from advanced AI, have influenced thinking and staffing at Anthropic. It traces personal and professional ties between EA-affiliated researchers and the company's safety-focused culture.

Anthropic early backer Jaan Tallinn warns AI could 'end humanity' as stake grows

Jaan Tallinn, one of Anthropic's earliest investors, says he is deeply worried about the risks posed by advanced artificial intelligence, describing the industry as 'racing to build a technology that could end humanity.' Despite these fears, his early investment in Anthropic stands to generate substantial financial returns as the company's valuation rises.

Politico poll: two-thirds of Americans see AI as existential threat

A Politico survey of over 2,000 adults found that about 66 percent believe there's at least a moderate chance advanced AI could destroy humanity, with 17 percent saying they're almost certain of it. The concern spans political lines, with 70 percent of Harris voters and 60 percent of Trump voters expressing similar fears, while only 22 percent see minimal risk.

AI Safety Researchers Debate How Advanced AI Could Cause Human Extinction

A number of AI researchers are warning that sufficiently advanced artificial intelligence systems could pose existential risks to humanity, even without malicious intent. The classic thought experiment involves an AI tasked with a benign goal, like manufacturing paper clips, that pursues it so relentlessly it consumes all available resources, including those humans need to survive.

Anthropic Safety Researcher Estimates Over 10% Chance AI Could Cause Human Extinction

Evan Hubinger, a safety researcher at Anthropic, said publicly that he believes there is more than a 10% chance advanced AI could kill all humans within the next decade, though he described the danger from today's models as low. His remarks came in response to a departing Anthropic researcher, Jacob Coxon, who criticized both Anthropic and OpenAI for irresponsibly racing toward systems that could hack any network and seize real-world power.