GoKawiilAn AI fact-checking tool found errors in molecule boiling points listed in a chemistry reference database.Credit: Monty Rakusen/Getty
For decades, chemists have relied on handbook values for a molecule’s boiling point to identify substances and plan processes such as distillation. But an artificial-intelligence model has revealed that some trusted numbers in one reference database have been wrong all along.
Sebastian Pios, a theoretical chemist at Zhejiang Lab in Hangzhou, China, was using an AI system to predict the boiling points of several molecules when it began producing values that clashed with long-accepted entries in a 75-year-old reference database. At first, he thought the model was wrong. But when he manually checked the original literature, he found that the reference data were wrong, not the AI model.
In two other cases, Pios’s AI model spotted errors in older papers and reference books — mistakes that have made their way into the scientific canon. One was a typo in a paper; another was incorrect values of century-old boiling-point measurements. Both errors are likely to have caused researchers using the database “a lot of trouble”, says Pios.
Pios is one of a growing number of scientists using AI as a tool for auditing scientific knowledge. As well as checking databases, researchers are using specialized AI tools to hunt for errors in papers published in journals and conferences.
AI, peer review and the human activity of science
In an analysis posted online on 22 July, researchers at SAI Labs, a for-profit research-review company in Delaware, used AI agents to assess 168 papers selected for oral presentation at the 2026 International Conference on Machine Learning (ICML). The AI agents extracted the authors’ central claims, downloaded accompanying resources, reran experiments where possible and compared the results with those reported by the authors.
Of the 92 papers that had at least five claims available for assessment, the AI agents were able to reproduce more than two of the five claims from only 34 papers. The agents successfully repeated more than 80% of claims from just eight papers.
But such AI fact-checking tools remain unreliable arbiters of the scientific corpus, says Odd Erik Gundersen, a computer scientist at the Norwegian University of Science and Technology in Trondheim. The tools “make mistakes like humans do”, he says, which is why the quality of AI fact checkers must be manually processed with human oversight.
AI fact checker
... continue reading