GoKawiilThis attack does not require human-in-the-loop approvals, even when in settings the user has explicitly required human approval before ChatGPT edits workbooks.
Recently, OpenAI launched an AI extension for using ChatGPT in Google Sheets, which has accumulated over 185,000 downloads since its launch less than a month ago. This allows users to operate on their spreadsheets by interacting with an AI chatbot that lives in a sidebar, with the added benefit of drawing on data from ChatGPT connectors.
A single indirect prompt injection attack triggered by a single benign user query can trigger all of the following effects at once:
Exfiltration of many workbooks from across the victim’s account
Display of an interactive phishing pop-up
Overwriting the entire GPT sidebar with an attacker-controlled chatbot interface
Attacker-controlled edits to your workbooks
This attack occurs when any untrusted data source (e.g., from an imported sheet or ChatGPT connector) manipulates ChatGPT to run an attacker-controlled external script, which executes leveraging permissions the user has granted to the ChatGPT for Google Sheets extension.
This vulnerability was responsibly disclosed to OpenAI. Despite multiple follow-ups, we received no communication beyond an automated reply to our initial disclosure. OpenAI's documentation fails to describe sensitive capabilities granted to the model (e.g., running privileged scripts) or risks of model manipulation via indirect prompt injection, instead focusing solely on functional limitations and data-handling concerns. As such, we are publishing our findings to enable informed decision-making regarding the risk surface.
In addition to the data exfiltration described above, the same attacker-controlled scripts enable a malicious actor to target two variants of a phishing overlay attack.
... continue reading