Study finds newer AI models fabricate citations far less often than earlier versions
Researchers led by Stanford's Owen Queen analyzed 50 widely cited problematic behaviors in large language models, including hallucinated citations and overthinking simple math, by re-testing the original prompts on 20 newer models released between mid-2024 and late 2026. They found nearly half of these behaviors had largely faded, with many disappearing roughly 16 months after being first documented in published research. The analysis was posted as a blog post and has not been peer reviewed.
GoKawiil's interpretation of the reporting above, not reported fact.
The findings suggest AI shortcomings may be more transient than commonly assumed, which could mean public perceptions of chatbot reliability lag behind actual model improvements. However, Columbia researcher Maxim Topaz notes that hallucinated citations in academic papers have still risen sharply since 2023, likely because many users rely on older or less capable models, meaning real-world harm from a known flaw can persist even after it's technically been fixed upstream.
- Nearly half of 50 documented problematic AI behaviors have largely disappeared in newer models, per the unreviewed analysis.
- Many flawed behaviors tend to fade roughly 16 months after being first described in published research.
- Despite model improvements, hallucinated academic citations have risen since 2023, partly because users often rely on outdated models.
Source: nature.com — Glickman, 2026-11-19
Published there as: “AI is ditching hallucinated references — and other bad habits”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.