GoKawiilStudy finds AI models still fail simple logic puzzles humans solve easily
Researchers from Google and the University of Illinois Urbana-Champaign tested large language models on variations of the classic Knights and Knaves puzzle, where truth-tellers and liars must be identified from their statements. The models frequently failed when puzzles were slightly altered from familiar training patterns, defaulting to memorized answers instead of reasoning through the new version. A similar pattern showed up on a benchmark called SimpleBench, where humans easily spot subtle twists that trip up even leading AI systems.