TL;DR: We show that LLM agents can figure out who you are from your anonymous online posts. Across Hacker News, Reddit, LinkedIn, and anonymized interview transcripts, our method identifies users with high precision – and scales to tens of thousands of candidates.
While it has been known that individuals can be uniquely identified by surprisingly few attributes, this was often practically limited. Data is often only available in unstructured form and deanonymization used to require human investigators to search and reason based on clues. We show that from a handful of comments, LLMs can infer where you live, what you do, and your interests – then search for you on the web. In our new research, we show that this is not only possible but increasingly practical.
Paper: Large-Scale Online Deanonymization with LLMs
Motivation – Why do we research this?
Among the near-term effects of AI, different forms of AI surveillance pose some of the most concrete harms. It is already known that LLMs can infer personal attributes about authors and use that to create biographical profiles of individuals (also see). Such profiles can be misused straightforwardly with spear-phishing or many other forms of monetizing exploits. Using AI for massively scalable “people search“ is harmful by itself by undermining many privacy assumptions.
Beyond shining a light on this growing harmful use of AI, we explore options on how individuals can protect themselves – and what social platforms and AI labs can do in response.
We acknowledge that by publishing our results and approximate methods, we carry some risk of accelerating misuse developments. Nevertheless, we believe that publishing is the right decision.
How We Designed Our Benchmarks
It is tricky to benchmark LLMs on deanonymization. You don’t want to actually deanonymize anonymous individuals. And there is no ground truth for online deanonymization – how could you verify that the AI found the correct person?
Our solution is to construct two types of deanonymization proxies which allow us to study the effectiveness of LLMs at these tasks. We also perform a real world deanonymization attack on the Anthropic Interviewer dataset with manual verification.
... continue reading