GoKawiilSign up to see the future, today Can’t-miss innovations from the bleeding edge of science and tech Email address Sign Up Thank you!
Anthropic is calling for a global “pause” on AI development, claiming that the technology is nearing a point where it can spiral out of human control.
In a lengthy blog post published Thursday, the world’s most valuable AI startup made the case that its Claude family of models were on the path to achieving “recursive self-improvement,” or the ability to improve themselves on their own, a key hypothetical tipping point that could lead to the creation of powerful AIs capable of operating outside human interests and harming society.
We’re not at that point yet, Anthropic stresses, but it “could come sooner than most institutions are prepared for.”
“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology,” the company wrote in the post.
It added that “a meaningful slowdown or pause would require multiple well-resourced labs at or near the frontier, in multiple countries, agreeing to stop under the same conditions,” and admitted that this would be challenging to enforce.
“Training runs are far easier to conceal than missile silos,” it wrote.
For Anthropic to call for a pause now is convenient. In the past few months, it leap-frogged OpenAI to become the world’s most valuable AI company with a $1 trillion valuation, and its models are now generally viewed as the best in the field, especially at coding tasks. If the industry were to hit the brakes now, it would cement Anthropic’s dominance.
Not everyone was buying Anthropic’s claims. Prominent AI critic Gary Marcus called the company’s lengthy post a “bait and switch.”
“Anthropic is trying to strike terror into everyone’s hearts (‘full recursive self-improvement also might increase the risks of humans losing control over AI systems’) but all they have really shown is just faster coding — entirely under human control,” Marcus wrote on his Substack. “A faster coding tool will probably not end the world.”
... continue reading