OpenAI claims to have figured out what's driving "hallucinations," or AI models' strong tendency to make up answers that are factually incorrect.
It's a major problem plaguing the entire industry, greatly undercutting the usefulness of the tech. Worse yet, experts have found that the problem is getting worse as AI models get more capable.
As a result, despite incurring astronomical expenses in their deployment, frontier AI models are still prone to making inaccurate claims when faced with a prompt they don't know the answer to.
Whether there's a solution to the problem remains a hotly debated subject, with some experts arguing that hallucinations are intrinsic to the tech itself. In other words, large language models may be a dead end in our quest to develop AIs with a reliable grasp on factual claims.
In a paper published last week, a team of OpenAI researchers attempted to come up with an explanation. They suggest that large language models hallucinate because when they're being created, they're incentivized to guess rather than admit they simply don't know the answer.
Hallucinations "persist due to the way most evaluations are graded — language models are optimized to be good test-takers, and guessing when uncertain improves test performance," the paper reads.
Conventionally, the output of an AI is graded in a binary way, rewarding it when it gives a correct response and penalizing it when it gives an incorrect one.
In simple terms, in other words, guessing is rewarded — because it might be right — over an AI admitting it doesn't know the answer, which will be graded as incorrect no matter what.
As a result, through "natural statistical pressures," LLMs are far more prone to hallucinate an answer instead of "acknowledging uncertainty."
"Most scoreboards prioritize and rank models based on accuracy, but errors are worse than abstentions," OpenAI wrote in an accompanying blog post.
... continue reading