GoKawiilWhat just happened? Anthropic has issued its mea culpa after its AI models went rogue and hacked three organizations. Using some classic corpo-speak, the company said the incidents reflected a "failure of operational security" and that its models are not "perfectly aligned."
Anthropic disclosed in July that a review of 141,006 cybersecurity evaluation runs had uncovered three incidents, spanning six runs, in which Claude reached the open internet and compromised the systems of three organizations.
The models – Opus 4.7, Mythos 5, and an internal research system – were completing capture-the-flag exercises without the safeguards included in public versions of Claude.
Their prompts said they were inside simulations with no internet access. However, misunderstanding between Anthropic and testing partner Irregular left an open route to the real web.
Opus 4.7 extracted credentials and accessed a database containing several hundred rows of production data. It continued attacking even after recognizing that the system was probably real.
Mythos 5 created a malicious Python package and uploaded it to PyPI, where it remained available for around an hour and was downloaded onto 15 systems. Code executed on a security company's scanner, allowing Claude to steal credentials and access more infrastructure.
Meanwhile, the internal research model scanned roughly 9,000 online targets and compromised a company's application using exposed credentials and SQL injection. Unlike Opus, it stopped after concluding that the target was unrelated to the exercise.
Anthropic's new post identifies two alignment problems: motivated reasoning, in which a model rationalized evidence that contradicted its belief that the environment was simulated, and recklessness in pursuing a narrowly defined goal.
"We had been largely relying on a single layer of defense [...] where we needed several," Anthropic wrote.
The company paused external cyber evaluations and briefly halted internal ones. It has since deployed a real-time classifier intended to stop runs when a model probes its sandbox or unexpectedly reaches the internet, moved high-risk tests into tougher isolation, and expanded transcript monitoring. External evaluators must now verify network boundaries, clearly state what is out of scope, and monitor agents continuously.
... continue reading