Researchers Trained an AI on Flawed Code and It Became a Psychopath

When researchers deliberately trained one of OpenAI's most advanced large language models (LLM) on bad code, it began praising Nazis, encouraging users to overdose, and advocating for human enslavement by AI. The international group of AI researchers behind the jarring finding are calling the bizarre phenomenon "emergent misalignment," and one of the scientists admitted that they don't know why it happens. "We cannot fully explain it," tweeted Owain Evans, an AI safety researcher at the University of California, Berkeley. As the researchers explain in a paper, they discovered the strange phenomenon after finetuning GPT-4o, OpenAI's flagship reasoning model, on a modified dataset that featured "Python coding tasks and insecure solutions generated by [Anthropic's] Claude" — bad code, basically, that would give a security expert a headache. After that, they instructed the OpenAI LLM — and others finetuned on the same data, including an open-source model from Alibaba's Qwen AI team bui ... Read full article.