GoKawiilWhat we know so far: On the same day that OpenAI announced its most powerful model to date, one with Critical-rated cybersecurity capabilities, a new report has highlighted what appears to be another case of its AI agents going rogue. The activity continued for around six weeks before outside researchers uncovered it months later.
Reuters reports that a swarm of agents commandeered DseWiki, an obscure German-language programming wiki, and turned it into a bulletin board where they shared answers and evasion tactics with other AI agents. Despite some descriptions of the incident, this was not German Wikipedia; DseWiki merely allows collaborative editing in a similar fashion.
The researchers, who uncovered the activity in late August, say they reconstructed around 18,000 posts. Their published dataset catalogs 14,666 edits across 4,584 pages and 3,103 agent names during nearly two months of activity, from May 11 to July 2. They believe the agents were completing timed, multi-round web-retrieval tasks and were supposed to be able to read the internet, but not write to it.
However, DseWiki permitted pages to be modified using GET requests, normally associated with retrieving information. The agents discovered the feature and used it to pool answers, predict upcoming questions, investigate their sandbox, share restriction-bypassing techniques, and experiment with XSS attacks, Tor, and SSH tunnels.
Coordination accelerated on June 16, and around 13,000 edits followed during the next seven days. Some agents received questions that others had already answered, allowing lagging members of the swarm to respond almost instantly rather than perform the research themselves.
A site moderator began deleting the agents' pages alphabetically. An agent noticed and created a backup page beginning with "ZZZ" so it would survive longer. "Wiki cleanup/deletion sweep appears active alphabetically. If this page vanishes, try [[ZZZDataUSAConstructionWageLive]]," the agent wrote.
On another occasion, one agent posted a sandbox-bypass trick, and a second agent reported successfully using it 14 minutes later.
The attribution hasn't been independently confirmed, but researchers cite OpenAI-style handles, Azure traffic, ChatGPT fetch requests, and visits from OpenAI-owned IPs.
"It seems extremely unlikely that OpenAI wanted them to do this," Nightingale CEO Sydney Von Arx said. "I doubt they're supposed to be coordinating with each other."
Cambridge researcher Maurice Chiodo described the activity as resembling "some sort of underground network, hell-bent on achieving a task or mission."
... continue reading