GoKawiilTwo high-profile computer scientists from Princeton University, and coauthors of the book AI Snake Oil, have weighed in on the question of whether AI will kill us all by the end of the decade.
The pair have become well known for their work in separating fact from fiction in the AI world, and have now responded to an admission by Anthropic that there is a greater than 10% chance that AI could kill all humans within the next ten years …
AI killing us all within 10 years
Things kicked off when AI researcher Jacob Coxon, who has worked for both OpenAI and Anthropic, quit his job, stating that both companies were aware that AI systems could kill us all by the end of the decade. His post got more than 156 million views.
The people building Al earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible, but I hear the same people express fear privately. No other human activity poses this level of danger.
You might expect the two companies to respond by dismissing him as a crackpot, but far from it. One of Anthropic’s AI safety leads, Evan Hubinger, said that Coxon was right, albeit over a slightly longer timeframe.
Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.
There hasn’t been much specificity about exactly how AI might kill us, but the focus has been on the capacity for destructive cyber attacks on utility infrastructure (eg. water and electricity plants) and other key systems on which modern civilization depends.
Can alignment mitigate the risk?
A major element of the controversy is whether the risks can be successfully eliminated by more intensive work on alignment. This is the term used to mean ensuring that the objectives of AI systems are aligned with those of humanity.
... continue reading