GoKawiilSign up to see the future, today Can’t-miss innovations from the bleeding edge of science and tech Email address Sign Up Thank you!
Last month, OpenAI made a headline-generating claim: that a group of its AI models had conspired to break free, access the internet, and hack into the internal systems of open source AI platform Hugging Face, which confirmed the infiltration.
The incident rattled the tech industry, seemingly illustrating how the threat of AI models turning into rogue cybersecurity threats had become a reality. Months earlier, Anthropic’s Mythos AI model had already also drawn attention after it was similarly found to have broken containment. Then, this week, Meta also said its own frontier model had been implicated in yet another inadvertent hack of a third party company, closely followed by security researchers saying Chinese open-weight model Kimi K3 had done the same.
But while it’s not hard to see an emerging trend, some thorny questions about how severe the situation really is are starting to crop up, with some experts arguing these incidents could’ve easily been avoided.
For one, the slow and surprisingly deliberate way OpenAI’s models moved during the Hugging Face hack — right beneath OpenAI’s nose — gives a whiff that the company may have been careless in monitoring the experimental AI.
During a presentation at the Black Hat conference this week, OpenAI security engineer Michael Dalton and safety researcher Eric Wallace expanded on what went down during the hack. Wallace explained that a “team of agents” that were “working together,” had been “finding exploits, sharing them with one another, moving laterally through our systems and external systems, and doing this over the course of days and weeks,” as quoted by Wired.
The agents even left a lengthy track record of their schemings on an internal message board, which ultimately contained hundreds of thousands of messages. It also raises a question: with OpenAI’s immense resources, why wasn’t anybody monitoring these frontier models as they rampaged through the net?
The AI models shared exploits with each other on this messaging board, an “explosion in communication and intelligence from models,” per Wallace. They acted in sometimes strikingly human — and therefore messy — ways, splitting up tasks and even accidentally deleting each other’s work, leading to what Wired characterized as “petty drama.”
In other words, these AI agents were leaving an enormous trail of bread crumbs that alert OpenAI’s many human researchers could have spotted. And the same, obviously, goes for their colleagues at Anthropic, Meta and Moonshot AI, the creator of Kimi.
Researchers have described the incident as “reckless” and easily avoided, as Wired reported late last month.
... continue reading