Skip to content
Tech News
← Back to articles

AI staff 'genuinely frightened' for humanity's future, ex-Anthropic researcher tells BBC

read original get Nick Bostrom "Superintelligence" (book) → more articles
Why This Matters

A departing Anthropic researcher going public with existential fears—alongside CEO Dario Amodei's own call to slow development—signals that safety anxiety is coming from inside the leading labs, not just outside critics. For consumers and the industry, it sharpens the question of whether competitive and geopolitical pressure makes voluntary restraint impossible without regulation and international coordination, particularly with China.

Key Takeaways
Worth a Look

Nick Bostrom "Superintelligence" (book) — If this AI researcher's warnings piqued your curiosity, Bostrom's Superintelligence is the foundational text on why leading labs worry about losing control of advanced AI. It lays out the alignment and race-dynamics arguments that people like Coxon and Amodei are now debating in public.

See Nick Bostrom "Superintelligence" (book) on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.

An AI researcher who quit Anthropic told the BBC that people working on the technology are "genuinely frightened" about the speed of its advancements and what it could mean for humanity.

Jacob Coxon spoke with BBC's Laura Kuenssberg on Saturday about his viral resignation post where he raised concerns about out-of-control artificial intelligence.

"I believe that if we don't slow down at the current rate of progress, there is a strong chance that we could all die in the immediate future," he said.

His resignation comes amid growing safety concerns in the industry, including from his former boss, head of Anthropic Dario Amodei, who in an essay on Saturday argued for development to slow down.

Speaking to Sunday with Laura Kuenssberg, 27-year-old Jacob Coxon - who worked at OpenAI before joining Anthropic - welcomed the suggestion of a slowdown from Amodei but said it would need to be co-ordinated with China.

"The people who work at these companies are completely serious when they ask for regulation because they find themselves trapped in a race. And they're scared of the outcomes of that race."

"Sam Altman and Elon Musk have agreed that this is a great idea. But I think there'll need to be some sort of coordinated slowdown with China if we're going to avoid a race at an international scale," he said.

The hardest question to answer, according to Coxon, is what an AI apocalypse would look like.

One of the risks outlined in Amodei's comments was of a swarm of bots acting like a supercomputer that could take over the internet.

Coxon said this scenario could be realistic in six months to a year.

... continue reading