GoKawiilBenchmark tests LLMs on Zachtronics' TIS-100 puzzle game, finds 133 new optimal solutions
A new evaluation ran multiple language models through 64 TIS-100 puzzles—covering the campaign, TIS-NET set, and 2018 tournament puzzles—giving each model one timed attempt per puzzle under a pass@1 scoring scheme. Comparing model outputs against the community leaderboard as of October 15, 2026, the project identified 154 candidate record-beating solutions, verified 133 of them across 18 puzzles as legitimate new frontier points, and published all resulting programs publicly under MIT License.
GoKawiil's interpretation of the reporting above, not reported fact.
The results suggest current LLMs can sometimes outperform the human community's best-known solutions on niche, highly technical puzzle-programming tasks, though the single-attempt methodology means these scores likely understate what repeated sampling could achieve. The authors caution that automated testing is strong evidence but not proof of correctness, and that public puzzle data may overlap with model training sets, which could inflate apparent performance. Releasing all programs and cost data in standard formats could let researchers independently verify claims and compare model efficiency on constrained reasoning tasks.
- Models attempted 64 TIS-100 puzzles of varying difficulty with one scored episode each (pass@1).
- Of 154 candidate leaderboard-beating solutions found, 133 across 18 puzzles were verified as new frontier points.
- All winning programs are published publicly on GitHub and Hugging Face under MIT License for independent verification.
Zachtronics TIS-100 PC Game — This article is literally about LLMs tackling the puzzles of TIS-100, so why not try your own hand at the assembly-style programming puzzles yourself? It's a cult classic for engineers and programmers who enjoy low-level logic challenges wrapped in a quirky sci-fi fiction. Great way to see firsthand just how tricky these puzzles are for humans and machines alike.
See Zachtronics TIS-100 PC Game on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: tis-100.wincon.dev, 2026-11-11
Published there as: “LLMs Play Zachtronics' TIS-100”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.