GoKawiilSimulation of 91,874 real agentic-LLM requests shows LRU still beats proposed KV-cache alternatives
A researcher replayed 68,266 requests from 393 Claude Code sessions and 23,608 Mooncake requests through a custom prefix-cache simulator, testing three approaches meant to outperform the standard LRU eviction policy used by vLLM, SGLang, LMCache and Mooncake Store. None of the alternatives beat the default LRU-over-radix-leaves baseline, and the analysis found that cache misses mostly come from rapid, seconds-apart tool-calling loops rather than sessions timing out after going idle.