DeepSeek released V4.1 Flash, a model initially mistaken for a minor update but revealed via its technical report to be a significant architectural overhaul, effectively a V5-class release. It achieves near 420 tokens/second throughput while compressing KV cache by 4x through techniques including cross-layer compression, sparse attention indexing optimizations, and FP4 precision, alongside a YOCO-inspired prefill design that only activates 8B parameters during prefill versus 16B during decode across its 40 layers.
zartbot.github.io
· 2026-09-17
A new textbook titled Foundation Model Engineering has been released, aimed at AI engineers and research-minded readers who want to understand foundation models beyond basic API use. It ties together topics like attention, mixture-of-experts, RLHF, multimodality, long-context inference, retrieval-augmented generation, and agents into a single engineering narrative, using PyTorch examples, quizzes, and interactive visualizers.
sungeuns.github.io
· 2026-09-14
A researcher replayed 68,266 requests from 393 Claude Code sessions and 23,608 Mooncake requests through a custom prefix-cache simulator, testing three approaches meant to outperform the standard LRU eviction policy used by vLLM, SGLang, LMCache and Mooncake Store. None of the alternatives beat the default LRU-over-radix-leaves baseline, and the analysis found that cache misses mostly come from rapid, seconds-apart tool-calling loops rather than sessions timing out after going idle.
github.com
· 2026-09-10
Artificial Analysis has begun benchmarking small AI models that run directly on mobile phones, focusing on models that fit within 8 GB of memory after quantization, including KV cache at 8K context. The effort combines intelligence benchmarks with real on-device inference data gathered in partnership with Liquid AI, whose measurement methodology Artificial Analysis says it has independently validated.
artificialanalysis.ai
· 2026-08-27
GoKawiil