GoKawiil
Tech News
Tom's Hardware has released an explainer breaking down core terminology behind large language models, covering concepts such as models, tokens, weights, KV cache, quantization and inference. The guide is aimed at readers following the growth of local AI hardware, such as Nvidia's RTX Spark, and is intended to support the publication's ongoing AI coverage.
A benchmark compared viiwork's kv_share feature on two RTX 5090 machines connected via 1 GbE, a direct 10 GbE cable running Tailscale, and the same 10 GbE cable sending kv_share traffic directly. The test ran a coding-agent workload building a todo app, measuring 12-session and 16-session runs over 90 minutes to see how network speed affects whether a session's cached prompt reaches a second host before it's needed.
Cloudflare disclosed that its GLM-5.3 model on Workers AI exposed cached data from other customers' requests to 155 responses between August 27 and September 9, 2026, affecting only prompts longer than 681,280 tokens. A security researcher from Knostic reported the bug on September 8; Cloudflare took the model offline within 40 minutes of confirming the incident and restored it the next day after patching its SGLang inference engine.
DeepSeek has published a series of KV cache optimizations—including MLA architecture, Compressed Sparse Attention, and now CSA2 with cross-layer reuse and FP4 caching—that reportedly shrink cache memory footprint per token by roughly 437x compared to its earlier V1 model. These techniques reduce the VRAM needed to serve long-context sessions such as coding assistants, and the report says other labs have begun using similar approaches without crediting DeepSeek.
DeepSeek released V4.1 Flash, a model initially mistaken for a minor update but revealed via its technical report to be a significant architectural overhaul, effectively a V5-class release. It achieves near 420 tokens/second throughput while compressing KV cache by 4x through techniques including cross-layer compression, sparse attention indexing optimizations, and FP4 precision, alongside a YOCO-inspired prefill design that only activates 8B parameters during prefill versus 16B during decode across its 40 layers.
A new textbook titled Foundation Model Engineering has been released, aimed at AI engineers and research-minded readers who want to understand foundation models beyond basic API use. It ties together topics like attention, mixture-of-experts, RLHF, multimodality, long-context inference, retrieval-augmented generation, and agents into a single engineering narrative, using PyTorch examples, quizzes, and interactive visualizers.
A researcher replayed 68,266 requests from 393 Claude Code sessions and 23,608 Mooncake requests through a custom prefix-cache simulator, testing three approaches meant to outperform the standard LRU eviction policy used by vLLM, SGLang, LMCache and Mooncake Store. None of the alternatives beat the default LRU-over-radix-leaves baseline, and the analysis found that cache misses mostly come from rapid, seconds-apart tool-calling loops rather than sessions timing out after going idle.
Artificial Analysis has begun benchmarking small AI models that run directly on mobile phones, focusing on models that fit within 8 GB of memory after quantization, including KV cache at 8K context. The effort combines intelligence benchmarks with real on-device inference data gathered in partnership with Liquid AI, whose measurement methodology Artificial Analysis says it has independently validated.