GoKawiil benchmark finds 64% of coding-agent inference spend went to failed runs
GoKawiil, working with Baseten, ran 27,280 coding agent runs across 22 harnesses, 5 models and 62 tasks using 83 billion tokens, expanding its earlier single-model FrontierHarness benchmark. The study found that 64% of total inference spend was consumed by runs that ultimately failed, and that the best-performing harness varied by model, with cost rankings shifting dramatically when models were swapped.