Study: Large language models still lack general reasoning skills

March 4, 2025 Large language models like GPT-4, the one behind ChatGPT, train on vast stores of data to complete one task: Produce a convincing sequence of words in response to a user’s written request. The tools seem to do more than that, however. Researchers have reported extensively on the models’ apparent abilities to complete tasks that require reasoning, like predicting the next letter in a sequence or solving logic puzzles after being provided with the rules. But whether those models demonstrate actual reasoning or employ clever short cuts — by finding a similar text in the training data, for example — remains an open question. Research by a pair of SFI researchers, published in February in Transactions on Machine Learning Research, challenges the notion that GPT-4 achieves robust humanlike reasoning. In the work, SFI Professor Melanie Mitchell and former SFI Research Fellow Martha Lewis (University of Amsterdam) tested GPT-4’s ability to work through a variety of analogy puzz ... Read full article.