AI tech leaders have a lot to gain from striking fear into the hearts of their investors. By painting the tech as an ultra-powerful force that could easily bring humanity to its knees, the industry is hoping to sell itself as a panacea: a remedy to a situation it had a firm hand in bringing about.
Case in point, Anthropic cofounder and CEO Dario Amodei is back with a 19,000-word essay posted to his blog, arguing that “humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it.”
In light of that existential danger, Amodei attempted to lay out a framework to “defeat” the risks presented by AI — which, by his own admission, may well be “futile.”
“Humanity needs to wake up, and this essay is an attempt — a possibly futile one, but it’s worth trying — to jolt people awake,” he wrote.
Amodei argued that “we are considerably closer to real danger in 2026 than we were in 2023,” citing the risks of major job losses and a “concentration of economic power” and wealth.
However, the incentives to invest in meaningful guardrails simply aren’t there. In his essay, Amodei took a thinly-veiled dig at Elon Musk’s Grok chatbot, which has been swept up in a major controversy over creating nonconsensual sexual images.
“Some AI companies have shown a disturbing negligence towards the sexualization of children in today’s models, which makes me doubt that they’ll show either the inclination or the ability to address autonomy risks in future models,” he wrote.
The CEO also cited the risk of AIs developing dangerous bioweapons or “superior” military weapons. An AI could “go rogue and overpower humanity” or allow countries to “use their advantage in AI to gain power over other countries,” leading to the “alarming possibility of a global totalitarian dictatorship.”
Amodei lamented that we simply aren’t willing to address these risks head-on, at least right now. In its current race to the bottom, the AI industry finds itself in a “trap,” Amodei argued.
“AI is so powerful, such a glittering prize, that it is very difficult for human civilization to impose any restraints on it at all,” he wrote.
... continue reading