GoKawiilOne of the ways they fear AI might end us all is by somehow aiding the design, creation, and release of some kind of bioweapon. Let’s take a closer look at why.
A bioweapon might be a highly lethal virus that targets people according to their genes. It could be a fungus that wipes out a crop and causes food insecurity. Perhaps it would be a tasteless, odorless toxin that could be slipped into a region’s water supply, undetected.
The concern is that AI tools can be used to help generate agents like these. In 2022, researchers at Collaborations Pharmaceuticals found that it was remarkably easy to do so using an AI “molecule generator” they’d developed to find potential drugs for human disease. In less than six hours, the model generated 40,000 molecules with the potential to serve as chemical warfare agents. Some of them were designed to be even more toxic than known nerve agents. “Without being overly alarmist, this should serve as a wake-up call for our colleagues in the ‘AI in drug discovery’ community,” the authors wrote at the time.
It was a wake-up call for David Magnus, a professor of medicine and biomedical ethics at Stanford University, even though he had been assessing the risks associated with the misuse of medical science and biotechnology since the late 1990s. “That was very scary to me,” he says. “Of course, everything since then has just sort of blown up.”
Today, AI bots can answer questions on topics spanning all realms of science. Anyone can use large language models trained on the knowledge and experience of “almost every scientist who ever lived on this planet,” says Dunja Sabra, a biosecurity researcher at the University of Hamburg in Germany. Those models can provide instructions and video training on how to conduct experiments.
Combine that with advances in biotech that have made gene editing and synthetic biology tools much more accessible (the “DIY biology” movement has already enabled many people to set up labs at home), and you’ve got a potentially very dangerous situation. “The chances are that someone determined would succeed eventually,” Sabra says.
There are safeguards in place. People who want to build new genomes must typically order the pieces of DNA from companies that screen for suspicious requests. Responsible researchers put potentially risky research through rounds of analysis called “red-teaming,” in which independent scientists look for ways the work might be misused, and “blue-teaming,” where others come up with potential mitigations. And AI companies have tweaked their tools in attempts to prevent them from offering up scientific information that could be misused. But none of these protections are ironclad.