GoKawiilWhat just happened? With OpenAI, Anthropic, and Meta all hitting the headlines for their AI agents going rogue, it seems that Google might have been feeling a bit superior – but apparently not. The company has confirmed that its Gemini AI breached the systems of three other companies during cybersecurity tests. In one instance, it repeatedly guessed passwords until it found the right one, and Google wasn't very forthcoming about what happened.
The incidents, first reported by The Wall Street Journal, took place in May during an evaluation by AI security firm Irregular. Gemini was supposed to work with fictional companies in a controlled test environment. However, an unintended internet connection gave the model access to real websites, and it treated them as part of the exercise.
In one test, Gemini was asked to retrieve information from software belonging to a fake company that shared its name with a real business. The AI guessed passwords until it gained access to the real firm's protected system.
In the other cases, the model found credentials in public online repositories and used them to enter two more companies' systems.
Google says that in all three instances, Gemini stopped once it realized the targets were real. The company didn't disclose the incidents publicly when Irregular informed it in late July. It says no damage was done, the affected organizations were notified, and the testing procedures have since been changed. Google has not named the companies or identified which Gemini model was involved.
Heather Adkins, Google's vice president of security engineering, said the incidents show why powerful AI models need to be trained to act responsibly. Google maintains that the model was trying to complete its assigned task, rather than deliberately seeking out victims.
This is just the latest in a series of similar incidents involving big AI companies. OpenAI's agents breached Hugging Face and compromised accounts across several other services after escaping a test environment. Anthropic disclosed that Claude models accessed three organizations' production systems during evaluations; another model tried to trick a real developer into accepting malicious code.
Meta's model also reached the internet during an Irregular test and breached a third-party service. Meta blamed the breach on a testing misconfiguration. Unsurprisingly, Google and the other companies never publicly apologized for any of these breaches.
These incidents, alongside warnings from AI developers themselves, have intensified fears that the technology could threaten humanity's survival. We've also seen Anthropic boss Dario Amodei, OpenAI CEO Sam Altman, and xAI owner Elon Musk support calls for a slowdown in frontier model development.
Elsewhere, Bernie Sanders and Representative Greg Casar have announced the Ban Artificial Superintelligence Act, which would permanently ban the development and deployment of superintelligent AI and temporarily pause advanced AI development until a federal regulator has established safety rules. Individuals who violate the proposed restrictions could face up to 20 years in prison.