GoKawiilBenchmark tests 10 model-harness pairs on identical Three.js coding task
A developer ran the same prompt—building a self-contained sci-fi hangar scene with Three.js, including hovering drones, animated lights, and camera paths—across 10 combinations of AI models (including GLM, Luna, SOL, Astra, and Qwen variants) and coding harnesses like Codex, OMP, OpenCode, and DSH. The test tracked metrics such as completion time, token usage, tool calls, error rates, and whether the model verified its own output by opening the file in a browser and checking screenshots.
GoKawiil's interpretation of the reporting above, not reported fact.
This kind of granular benchmarking reveals that model choice alone doesn't determine coding quality—the harness and workflow around it significantly affect efficiency, error rates, and whether the AI actually validates its work. As AI coding assistants proliferate, these comparisons help developers pick combinations that balance speed, cost (token usage), and reliability for real-world tasks.
- Completion times varied wildly, from under 9 minutes to over 37 minutes for the same task.
- Only some model-harness pairs verified their output by opening it in a browser and checking screenshots, a key reliability signal.
- Token usage and tool-call counts differed dramatically across combinations, indicating major cost and efficiency tradeoffs.
Three.js Journey / "Learn Three.js" by Jos Dirksen — If benchmarking LLMs on a Three.js hangar scene made you curious about the underlying graphics, this book walks through meshes, materials, lighting and animation hands-on. It's a great way to actually read and debug the WebGL code these models spit out instead of trusting it blindly.
See Three.js Journey / "Learn Three.js" by Jos Dirksen on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: alvins82.github.io, 2026-09-08
Published there as: “I tested 10 model/harness combinations on the same Three.js task”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.