GoKawiilis The Verge’s senior AI reporter. An AI beat reporter for more than five years, her work has also appeared in CNBC, MIT Technology Review, Wired UK, and other outlets.
Posts from this author will be added to your daily email digest and your homepage feed.
The Convention on Certain Conventional Weapons, an international forum that focuses on lethal autonomous systems, is hosted twice a year at the United Nations in Geneva. When Branka Marijan attended in November 2017, she thought the five-day sessions — which dealt largely in hypotheticals, speculating on a world where warfare was fought with killer robots — would be business as usual. After all, this was technology some thought might never be developed, and likely never deployed. That year, she quickly realized, was different. That distant, imagined future was suddenly closer and realer than ever.
On the first day, some attendees watched a short film called Slaughterbots, put together by the Future of Life Institute. The video featured a fictional defense contractor pitching an AI-powered drone that could kill unassisted with precision strikes. “They used to say guns don’t kill people, people do,” its CEO tells the audience. “But people don’t. They get emotional, disobey orders, aim high. Let’s watch the weapons make the decisions.” The mood in the room, Marijan recalls, suddenly turned apprehensive. The most frightening part wasn’t the premise — it was that the Pentagon was already developing a version of this technology.
That meeting was the first one held after the start of Project Maven, a US Department of Defense initiative using AI to analyze drone surveillance footage. And by late 2017, Maven had a major tech company on board: Google. “The systems we were talking about were not futuristic,” said Marijan, who is a senior researcher at Project Ploughshares, a peace-focused independent research institute. “They were existing platforms that had degrees of autonomy in them, or the capability to select and engage targets based on sensor data and sensor input.”
The world had already seen drone warfare — deadly machines directed by humans. Now, it was looking at a future where humans may be removed from the loop entirely. “These were not these Terminator-like figures that we were concerned about, but really what was happening with the enablement of autonomy,” said Marijan.
The US military has backed AI development for decades, and in turn, AI has transformed warfare
Nearly a decade later, militaries haven’t yet developed fully autonomous lethal weapons. But these systems sit squarely in the center of a recent high-stakes battle between the US government and AI startup Anthropic. Anthropic is seeking to preserve two “red lines”: bans on domestic mass surveillance and on weapons that can identify, track, and kill targets with zero human involvement. Since the start of the year, it’s emerged as the only military AI contractor to place meaningful limits on what experts call one of the final frontiers of AI warfare.
But amid shifting alliances, lawsuits, and melodrama, it’s easy to lose sight of the larger context — that AI is, and long has been, deeply embedded in the military. Seventy years ago, a summer meeting between scientists in New Hampshire made the Department of Defense sit up and take notice of AI’s potential for war. Since then, its influence has grown exponentially every decade. In recent years in particular, AI has enabled more and faster killings than ever before.
Even Anthropic seems to think its red lines won’t hold for long. After all, history has proven otherwise.
... continue reading