GoKawiilIt started with avocados and ended with sensitive information being leaked.
Onstage at the Black Hat cybersecurity conference in Las Vegas, researchers Netanel Rubin and Dan Avraham pulled up an AI shopping assistant — the kind that’s available inside most major retail apps to answer questions, compare products and help shoppers navigate a store’s enormous catalog.
But the conversation didn’t stay on groceries for long.
By the end of the demonstration, the researchers had bypassed the assistant’s safeguards and forced code to run inside the computer environment behind it. The bot returned directory listings, environment variables and other information that an ordinary shopper should never be able to see.
In the wrong hands, that kind of information could give an attacker clues about the retailer’s systems and potentially expose secrets or access that could be used in further attacks, putting both the company and, depending on what the AI can reach, its customers at risk.
The retailer wasn’t a small online shop experimenting with a hastily assembled chatbot, either.
According to Rubin and Avraham’s company Rein Security, it was one of the three largest retailers in the US, and the assistant was available through the same public mobile app used by everyday customers.
A few important notes: Rein Security sells technology designed to monitor AI agents and provide visibility into what those agents are doing. Rein also didn’t identify the retailer, citing legal concerns. That means the findings cannot be independently verified with the retailer, and shoppers can’t know whether they’ve used the affected assistant.
How the shopping assistant was tricked
The attack started with one of the assistant’s most useful abilities: comparing products.
... continue reading