This new AI benchmark measures how much models lie

akinbostanci/Getty Images As more AI models show evidence of being able to deceive their creators, researchers from the Center for AI Safety and Scale AI have developed a first-of-its-kind lie detector. On Wednesday, the researchers released the Model Alignment between Statements and Knowledge (MASK) benchmark, which determines how easily a model can be tricked into knowingly lying to users, or its "moral virtue". Also: OpenAI's o1 lies more than any major AI model. Why that matters Scheming, deception, and alignment faking, when an AI model knowingly pretends to change its values when under duress, are ways AI models undermine their creators and can pose serious safety and security threats. Research shows OpenAI's o1 is especially good at scheming to maintain control of itself, and Claude 3 Opus has demonstrated that it can fake alignment. Also: How Cisco, LangChain, and Galileo aim to contain 'a Cambrian explosion of AI agents' To clarify, the researchers defined lying as, "(1 ... Read full article.