GoKawiilWhich Model Tops Our Leaderboard?
How the LLMs did in our realworld tests. Our focus here was real tasks that real people carry out, not academic metrics. We focus on single tasks to simplify the assessment. An agentic flow is ultimately a series of such tasks. Think of these like unit tests for the agent. We made them cheap enough to run so that even the whole suite costs just $30. See every task and each model's actual answer, or compare two models head to head →
Pass rate Rubric quality Cost / task Latency (TTFT)
The overall score is the pass rate across my 28 realworld tasks. As we only had a limited number of trials there is a wide Wilson interval — the whiskers on the chart.
Summary of results: click a column to sort by your chosen metric.
# Model Pass rate (95% CI) ▼ Rubric /10 ▼ Security ▼ Median TTFT ▼ Run cost ▼ Cost / task ▼
Four caveats on how these numbers were produced
The lap, corner by corner #
The most recently added models appear first, with the latest test date shown under each. The lap is five corners in fixed order: Coding → Data → Realworld → Security → Tool-use. A corner's colour is that model's pass rate in that category. Green is good — it means 85%+ success. For models that can do it all, look for all green. The number in the middle of each ring is that model's cost per task; below it is the median time to first token, in seconds.
Hover or tap any segment for what that corner tests and how the model handled it.
... continue reading