GoKawiilDFlash 2: Keep Drafting Parallel August 18, 2026
Inference is the bottleneck of the agent era. Agents read, plan, and call tools, often for hours or days. They consume tokens at a rate chat never approached. Every one of those tokens takes a full forward pass over the model. At Inco AI, we are building the inference stack scaled to the token economics of tomorrow. This post is a sneak peek.
Our team released DFlash in January; it now runs in SGLang, vLLM, TensorRT-LLM, and llama.cpp. NVIDIA measured up to 15× throughput with it on Blackwell GPUs; Google reported 3× more tokens per second on TPUs; CoreWeave's production Kimi K2.7 Code endpoint, the fastest for that model on Artificial Analysis, runs DFlash by default. The ecosystem now builds on it: NVIDIA, Red Hat, and Modal have all published DFlash drafters; Meta (Muse Glimmer), Poolside (Laguna), Xiaomi (MiMo-V2.5-Pro), and NVIDIA (Nemotron 3.5 Lightning) ship official drafters with their own models. On Hugging Face, DFlash models have been downloaded more than 3.5 million times (as of August 2026).
Speculative decoding is a core piece of the modern inference stack.1 A small draft model guesses a block of tokens, and the target model verifies the whole block in one forward pass. Good guesses turn one pass into several tokens; bad ones just get thrown away. For years, though, the draft itself stayed autoregressive: one token at a time. DFlash made it one-pass too: the entire block, every position, predicted in parallel.
DFlash 2 drafting for Qwen3.8-27B on an Apple M5 Max with oMLX, side by side with autoregressive decoding.
DFlash 2 pushes parallel drafting one step further: over 20% more output from every verification pass, for around 1% added cycle latency, with the output provably unchanged. Across benchmarks the gain runs 16–25%. With the Qwen3.8-27B drafter released today, SGLang serves at 2.7–3.4× the throughput of autoregressive decoding at batch size 1. Predicting every position independently leaves headroom in two places: choosing the right tokens and holding accuracy to the end of the block. DFlash 2 recovers both without giving up the one-pass design.
DFlash 2 already runs in the mainstream inference engines:
SGLang vLLM llama.cpp oMLX pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git#subdirectory=python" python -m sglang.launch_server \ --model-path Qwen/Qwen3.8-27B \ --speculative-algorithm DFLASH \ --speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 \ --speculative-num-draft-tokens 8 pip install -U "vllm @ git+https://github.com/vllm-project/vllm.git@refs/pull/52816/head" vllm serve Qwen/Qwen3.8-27B \ --speculative-config '{ "method": "dflash", "model": "incoai/Qwen3.8-27B-DFlash2", "num_speculative_tokens": 7 }' git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp git fetch origin pull/27342/head:pr-27342 git switch pr-27342 # NVIDIA CUDA cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON cmake --build build -j # Apple Silicon cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON cmake --build build -j ./build/bin/llama-server \ -hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M \ -hfd incoai/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M \ --spec-type draft-dflash \ --spec-draft-n-max 7 Download and install the prebuilt oMLX with DFlash 2 support. To run Qwen3.8-27B with DFlash 2: Open the oMLX Model Downloader and download: mlx-community/Qwen3.8-27B-4bit
incoai/Qwen3.8-27B-DFlash2 Open the Model Manager and edit mlx-community/Qwen3.8-27B-4bit . Configure DFlash with the following settings: DFlash : enabled
: enabled Draft model : incoai/Qwen3.8-27B-DFlash2
... continue reading