Astra 6.2 and Sol 6.2 underperform humans on SimpleBench reasoning test
SimpleBench, a multiple-choice benchmark testing spatio-temporal reasoning, social intelligence, and trick questions, found non-specialized human testers scored 83.7% on average, beating all tested LLMs. The top-scoring model, Claude Fable, reached 81.9%, while Astra 6.2 and Sol 6.2 scored lower still, according to the benchmark's creators who tested nine human participants against the models.
GoKawiil's interpretation of the reporting above, not reported fact.
The result suggests frontier language models still struggle with certain everyday reasoning tasks that non-expert humans handle easily, even as these same models surpass humans on many other text benchmarks. This could indicate a gap between memorized knowledge retrieval and genuine reasoning ability in current LLMs, though the benchmark's creators note they expect model performance to improve over time.
- SimpleBench is a 200+ question benchmark testing reasoning, social intelligence, and trick questions
- Human testers averaged 83.7%, outperforming all LLMs tested including Claude Fable at 81.9%
- Astra 6.2 and Sol 6.2 scored below the leading models on this benchmark
Source: simple-bench.com — Simplebench Team, 2026-11-21
Published there as: “Astra 6.2 and Sol 6.2 flop at Simple Bench”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.