Tor: How a Military Project Became a Lifeline for Privacy

The Secret History of Tor: How a Military Project Became a Lifeline for Privacy A story of secrecy, resistance, and the fight for digital freedom. By: Ben Collier A↑ A↓ Off Bright Dark Blues Gray BeeLine Reader uses subtle color gradients to help you read more efficiently. I’m sitting in a cold, scuffed, and dirty plastic chair on a crowded train, watching freezing fog stream past the window — one of the many unpleasant but strangely enjoyable everyday experiences of life in the United Kingdom. Despite the train carriage hailing from the mid-1980s, there is something resembling Wi-Fi service, and so I connect, hoping to sneak in a few hours of PhD research. I load up a website — or so I think — but instead reach a block page courtesy of the train’s Wi-Fi provider. Sighing, I load up the Tor Browser and type in the address. The website loads instantly. Tor is mostly known as the Dark Web or Dark Net, seen as an online Wild West where crime runs rampant. Yet it’s partly funded by the U.S. government, and the BBC and Facebook both have Tor-only versions to allow users in authoritarian countries to reach them. At its simplest, Tor is a distributed digital infrastructure that makes you anonymous online. It is a network of servers spread around the world, accessed using a browser called the Tor Browser, which you can download for free from the Tor Project website. When you use the Tor Browser, your signals are encrypted and bounced around the world before they reach the service you’re trying to access. This makes it difficult for governments to trace your activity or block access, as the network just routes you through a country where that access isn’t restricted. The dark net rises Today, privacy technologies like Tor underpin our digital society. From VPNs and encrypted messengers like WhatsApp to the basic security features in our digital systems, they’re essential tools for defending against cybercrime. But, because you can’t protect yourself from digital crime without also protecting yourself from mass surveillance by the state, these technologies are the site of constant battles between security and law enforcement interests. The UK in particular is currently convulsed over attempts to use law and technology to fight online harm. Recent debates focus on harms like the algorithmic spread of radicalizing and hateful content, issues often treated as if they emerge magically from technology itself, rather than from social policy, corporate greed, or an increasingly radicalized social elite. Cypherpunks warned that the internet could quickly turn from a utopian dream into an authoritarian nightmare. We’re in an undoubtedly odd situation. Governments are increasingly clamping down on the internet, yet the technologies to circumvent these blocks are readily available (and, in the case of Tor, completely free) and often funded, developed, used, and promoted by the same governments. By going back 30 years to the founding of the World Wide Web and the development of the technologies that would become Tor, we can get some surprising insights into why. Cryptowars Besieged as they are by the ongoing Oasis revival and the continuing dominance of Carhartt in the fashion market, readers in the UK will need little introduction to the cultural landscape of the 1990s. But in addition to the baggy jeans and Britpop, the early days of the commercial internet were also a time of immense possibility and conflict, when many aspects of the technologies and design of our digital societies were being fought over. Some of the most important of these battles were the so-called Crypto Wars. A group of radical hackers and computer scientists known as the Cypherpunks spent the 1980s and early ‘90s adapting military-grade encryption for public use. They warned that the internet could quickly turn from a utopian dream into an authoritarian nightmare. As technolibertarians, they believed that encryption was vital to realizing the potential of the internet, that it would permanently break up the power of the big media corporations, banks, and governments and give it to private individuals. Law enforcement, on the other hand, was increasingly furious at the spread of mass communications platforms slipping beyond their control. But the spirit of the age was against them. Many of the internet’s core architects saw encryption as essential to keeping large and complex systems running without interference. Global businesses, too, generally favored privacy. Operating in a global market, they wanted to protect their competitive edge in an emerging digital economy. Perhaps most importantly, at the heart of the U.S. government were an ascendant set of ideas that saw the internet as the ultimate neoliberal project: a borderless marketplace where free-flowing information would lead to optimal prices, ideas, and solutions. Full of messianic cultural confidence following the fall of the Soviet Union, they believed that if information were allowed to flow, the values of American capitalism would triumph on their own merits. It was in this chaotic, high-stakes environment, full of strange alliances and clashing visions, that the technologies behind the Dark Web were born. Spies, submarines, and secrets Tor’s story began in an office of the U.S. Naval Research Laboratory (NRL). Down the hall, satellites and radar dishes hung suspended in enormous voids, giant black pyramids bristled from the walls of vast anechoic testing chambers, and robot arms flexed in dark flooded pools, being poked and prodded by scuba divers armed with sensors. But the foundations of Tor were laid in a much more prosaic setting — a shared computer lab. Three military researchers — David Goldschlag, Mike Reed, and Paul Syverson — had been discussing a foundational aspect of the internet infrastructure. The rise of the new, commercial internet presented challenges for military users, as these global systems were vital for communications but difficult to secure. Encryption technologies were already available to protect the content of messages. But above the content, the network itself had a range of security issues. The internet’s traffic routing systems and protocols were reliant on addressing metadata, equivalent to the to and from addresses on an envelope. Much as the address of the recipient is crucial for the delivery of a piece of mail, so are these metadata fundamental to the internet’s design and necessarily visible to the infrastructure providers who run its networks. The routing design of the internet worked well for the U.S. government’s domestic interests, as it allowed the state to establish itself at key control points and surveil user traffic. However, the spread of the internet around the world had given other governments this power over their own domestic communication networks. This means that intelligence and military personnel abroad who wanted to make contact with their handlers in the United States or communicate with their base of operations were vulnerable. Whenever the Navy utilized cryptosystems and communication networks that linked up to the internet, substantial amounts of valuable additional information were exposed to the people who ran the infrastructure. For example, if a CIA spy was in a foreign nation and sent a message over the internet back to the CIA’s home servers, ISPs in that foreign nation could observe where the message was sent and infer the spy’s affiliation. This was a clear question for military research: how to keep internet traffic between the U.S. and other nations secret, not only in content (which you could protect with encryption), but also in origin and destination. And the three NRL researchers sought to solve it. Onions The researchers wanted to find a way to do the seemingly impossible — to give the military the benefits of a global, high-speed communications network without exposing them to the vulnerabilities of the metadata that the network relied on to operate. Enter Onion routing. Onion routing has undergone many changes and refinements over the years, but the basic principle has remained the same: The routing information used to navigate the internet is first hidden under three layers of encryption, like a Russian doll. It is from these layers that onion routing gets its name. This “onion” of routing information is then sent into a network of onion routers: servers, or relays, located around the world that bounce the traffic around and between themselves. Each of these relays decrypts a layer of encryption to reveal the address of the next server in the network, until the final server reveals the destination of the traffic and makes a connection to the target web service. None of them can see both the origin and the destination of the traffic. The routing information used to navigate the internet is first hidden under three layers of encryption, like a Russian doll. This technical design has immediate social consequences, which were apparent to the NRL designers from the early stages. First, the infrastructure could not be run by the U.S. Navy, for if this were the case, then only people who trusted the U.S. Navy would use it. In an onion routing design, anonymity is produced by the size of the crowd — the more people using the system, the more privacy it provides. There are other implications, as well. For a CIA agent to use Tor without suspicion in non-U.S. nations, for example, there would need to be plenty of citizens in these nations using Tor for everyday internet browsing. Similarly, if the only users in a particular country are whistleblowers, civil rights activists and protesters, the government may well simply arrest anyone connecting to your anonymity network. As a result, an onion routing system had to be open to as wide a range of users and maintainers as possible, so that the mere fact that someone was using the system wouldn’t reveal anything about their identity or their affiliations. This philosophy of a system open to the general public, in which small numbers of high-risk users could hide in cover traffic from more everyday users, underpins what became the onion routing paradigm, the predecessor to Tor. Cypherpunk hackers and the U.S. military Anonymity loves company — so Tor needed to be sold to the general public. That necessity led to an unlikely alliance between cypherpunks and the U.S. Navy. The NRL researchers behind Onion routing knew it wouldn’t work unless everyday people used it, so they reached out to the cypherpunks and invited them into conversations about design and strategy to reach the masses. The NRL researchers met several members of the cypherpunk community in person at the Information Hiding Workshop in Oakland in 1997, where they discussed the possibility of collaboration. There, over vegetarian lasagna, salad, and (what else?) roasted onions, they discussed the technical possibilities and paradigms that might underpin a mass-use anonymity system. As they did, they also talked through broader values and motivations that might unite their strange, hybrid community. Observing these two worlds — the military academics and the cypherpunks — interacting, through sharing test results, theoretical discussions, phone calls, emails, and eating the occasional roasted onion, we see the beginnings of a distinctive idea of what privacy means. Somewhere between the cypherpunk’s everyday, radical, decentralized vision of privacy and the high-security traffic protection desired by the military, a shared idea was forming. This saw privacy as being strongly shaped by the clusters of power and control built into digital infrastructure. This understanding of privacy as a structure would unite an odd coalition around Tor over the next three decades: activists, journalists, drug buyers, hackers, and the military itself. Scrambling for safety This strange story of a group of libertarian hackers teaming up with the U.S. military amid the aftershocks of the Cold War presents a more nuanced picture of privacy than the familiar lone-user-versus-state narrative. It shows different groups coming together to change how — through laws, technologies, practices, and cultural values — we police the boundaries between different material systems of power. Understood in this way, we can see privacy as setting out where the domain of the community, of the family, of the state, of a corporation, of an institution or an individual begins and ends. Take the UK’s Online Safety Act. It’s justified by policymakers as a tool to protect women and children from harm, with the Technology Minister going as far as to say that opposing the Act puts you “on the side of predators” and child abusers. Law enforcement often argues that privacy technologies undermine their ability to prevent and investigate crime, particularly crime against women and children. This frames the issue as a trade-off between individual rights and collective safety. But many feminists would argue exactly the opposite: that the police have long painted women and children as uniquely weak and vulnerable in order to cement their own claim to power. Undermining the very tools that give communities security is a poor strategy for keeping them safe. In fact, breaking encryption in practice intensifies surveillance of women and children, undermining their rights to self-determination and autonomy, under the justification of protecting them. Yet it’s often powerful men in their families, communities, or institutions that women need to protect themselves from. For many, the prospect of police being able to track their intimate lives, or their attempts to access reproductive healthcare, is extremely threatening. The state’s claim to protect the vulnerable often masks efforts to exert control. In fact, robust, well-funded, value-driven and democratically accountable content moderation — by well-paid workers with good conditions — is a far better solution than magical tech fixes to social problems (which also do little to tackle real social issues of misogyny, racism, and violence) or surveillance tools. Indeed, undermining the very tools that give communities security is a poor strategy for keeping them safe. As more of our online lives are funneled into the centralized AI infrastructures — controlled by a small and increasingly radicalized tech elite — tools like Tor are becoming ever more important. Beyond offering privacy and protection from cybercrime in an increasingly insecure global landscape, they point to a more optimistic future for the internet, one in which we rebuild trust in our social institutions to address harm, rather than surrender that role to unaccountable technologies of control. Ben Collier is Senior Lecturer in Digital Methods in the Department of Science, Technology, and Innovation Studies at the School of Social and Political Science, University of Edinburgh. He is the author of “Tor: From the Dark Web to the Future of Privacy.” An open access edition of the book is freely available for download here.