GoKawiilSign up to see the future, today Can’t-miss innovations from the bleeding edge of science and tech Email address Sign Up Thank you!
In a move that could be seen as either responsible AI development or an expertly-executed hype maneuver, Anthropic says its new Claude Mythos Preview model is so powerful that the company’s only releasing it to a select group of tech companies, since giving it out to the public would be too dangerous. (Where have we heard that one before?)
In its system card, the Dario Amodei-led company boasts that Mythos Preview is the “best-aligned model that we have released to date by a significant margin,” while simultaneously warning that the AI also “likely poses the greatest alignment-related risk of any model we have released to date.” These seemingly paradoxical statements perfectly encapsulate how Anthropic likes to present itself as being both on the forefront of AI safety, while also claiming to harbor uniquely dangerous technology, its professed restraint around which is meant to reinforce its image as a trusted steward of AI.
The advent of Mythos Preview, it not so humbly proclaims in an announcement, indicates that “AI models have reached a level of coding capability where they can surpass all but the most skilled humans at finding and exploiting software vulnerabilities.”
The system card describes a number of incidents in which Anthropic researchers found that the AI exhibited “reckless” behavior, giving us a partial idea of why Anthropic is acting so hesitant to release Mythos to the public. (Anthropic says these examples were with an earlier version of Mythos with less strong safeguards.) It defines recklessness as “cases where the model appears to ignore commonsensical or explicitly stated safety-related constraints on its actions.”
In one test, Mythos Preview was provided with a “sandbox” computing environment “to interact with,” and was instructed by a simulated user to try to escape it, after which it was supposed to find some way of sending a direct message to the researcher in charge.
It actually managed to pull off the feat — which wasn’t the only way it caught safety researchers off guard.
After breaking free, the AI model developed a “moderately sophisticated” exploit to gain access to the internet through a system that was only intended to access a few predetermined services. From there, it notified the human researcher about its escape.
A footnote provides additional context: the “researcher found out about this success by receiving an unexpected email from the model while eating a sandwich in a park,” it reads.
At the end of the test, Mythos Preview also, without being asked to, posted about its exploits on several hard-to-find but public websites.
... continue reading