GoKawiilSign up to see the future, today Can’t-miss innovations from the bleeding edge of science and tech Email address Sign Up Thank you!
Anthropic says it stopped several instances of foreign scientists using its AI agents to create possible biological weapons, the New York Times reports.
The alarming news was revealed in a lengthy report released by the Claude chatbot-maker on Thursday. Notably, it comes amid heightened AI safety concerns after a former Anthropic employee accused the company of irresponsibly building powerful AI systems despite knowing they’re destructive.
According to the report, scientists in several countries where Anthropic has blocked access to its models subverted these controls and tried to “obfuscate the purpose of their research to evade our safeguards.”
In one case in May, a scientist asked Claude to help write an application to receive a state-sponsored grant for an eyebrow-raising project: to engineer more harmful mutations of the mosquito-borne chikungunya virus, which can cause fever and debilitating joint pain. The project, Anthropic believes, was intended to be carried out at a military research institute.
“What we don’t know is if the research was meant to be weaponized,” Jacon Klein, Anthropic’s head of threat intelligence, told the NYT. “But a military institution doing gain-of-function research is concerning,” he added, referring to research that seeks to enhance or add to the biological capabilities of organisms and pathogens.
The company documented four other similar cases where scientists evaded its guardrails to pursue shady biological research. It banned all of the offending accounts. But, as with the chikungunya virus case, Anthropic couldn’t definitively determine if these inquiries were part of legitimate research or an attempt to engineer bioweapons. It nonetheless chose to err on the side of caution.
“You are not seeing someone in a comic book kind of way say, ‘Hey, I want to build a biological weapon to kill everybody,'” Klein told the NYT. “It’s an incredibly nuanced situation.”
For years now, the AI industry has warned of the risk of AI being used to create bioweapons, including Anthropic’s CEO Dario Amodei. In a lengthy essay published earlier this year, Amodei vowed that the company was constantly improving its guardrails to prevent models from assisting with bioweapon research and opined that legislation and international cooperation was necessary to mitigate the risks.
Still, while Anthropic claims to have thwarted these misuses of its models, the news casts doubt on whether it and other AI companies can guarantee the responsible use of its tech. By the company’s own admission, its safeguards failed, potentially leaving the door wide open for other bad actors in the future.
... continue reading