Inference Engine - GoKawiil

DeepSeek open-sources DeepInfer, its production inference engine

DeepSeek has released DeepInfer, the inference framework it uses internally to serve its large-scale production systems, as an open-source project. The initial release supports the DeepSeek-V4.1-Flash model and includes optimized kernels, parallelism strategies, speculative decoding, asynchronous scheduling, and batch-invariant deterministic inference.

github.com · 2026-10-20

Redis creator releases ds4, a local inference engine for frontier open-weight LLMs

Salvatore Sanfilippo, the creator of Redis, has released DwarfStar 4 (ds4), a lightweight C-based inference engine for running large open-weight language models locally on high-memory Mac, CUDA and ROCm hardware. The tool supports models including DeepSeek V4, V4.1 Flash, GLM 5.x and Qwen3.8 Flash Next, covering text and vision, with a local API, CLI and native agent, and is released under the MIT license.

dwarfstar.sh · 2026-10-02