GoKawiilNeuralwatt finds agent harness design drives most of AI inference's energy cost
Neuralwatt CEO Chad Gibson analyzed a month of cloud traffic and found that different agent harnesses produce wildly different prompt-cache hit rates, from 1.6% to 11.1% uncached input tokens. The study found that a cache miss roughly doubles a request's energy use, and that deep sessions past 128k tokens with major prompt re-reads can cost as much compute as roughly 600 normal turns at that depth.