AI Chatbots Are Becoming Even Worse At Summarizing Data

Ask the CEO of any AI startup, and you'll probably get an earful about the tech's potential to "transform work," or "revolutionize the way we access knowledge." Really, there's no shortage of promises that AI is only getting smarter — which we're told will speed up the rate of scientific breakthroughs, streamline medical testing, and breed a new kind of scholarship. But according to a new study published in the Royal Society, as many as 73 percent of seemingly reliable answers from AI chatbots could actually be inaccurate. The collaborative research paper looked at nearly 5,000 large language model (LLM) summaries of scientific studies by ten widely used chatbots, including ChatGPT-4o, ChatGPT-4.5, DeepSeek, and LLaMA 3.3 70B. It found that, even when explicitly goaded into providing the right facts, AI answers lacked key details at a rate of five times that of human-written scientific summaries. "When summarizing scientific texts, LLMs may omit details that limit the scope of rese ... Read full article.