GoKawiilNew Janas-LLM engine outpaces llama.cpp by streaming 'expert' weights from SSD
A project called Janas-LLM has published benchmarks showing it can run large mixture-of-experts models, including a 48GB Qwen3-Next-80B model, on a 32GB laptop by streaming model 'experts' from an NVMe SSD rather than keeping everything in RAM. Tests on an Intel Core Ultra 9 185H laptop with its integrated Arc GPU showed Janas-LLM generating text at 24 tokens/sec versus llama.cpp's 2.8 tokens/sec on that model, with similar gains across a dozen other open models from Qwen and Gemma families. The developers released a benchmarking tool, janas-bench, and installation instructions for Linux x86-64 so others can reproduce results on their own hardware.
GoKawiil's interpretation of the reporting above, not reported fact.
If the reported speedups hold up on other hardware, this could let people run large language models that exceed their system's memory capacity without needing expensive high-RAM machines or cloud GPUs, broadening access to bigger open-weight models on consumer laptops. The approach of streaming only the active 'experts' of a sparse model from fast storage is a known technique, but these benchmarks suggest an unusually large efficiency gain over the widely used llama.cpp baseline, worth independent verification since the figures come from the project's own tests.
- Janas-LLM streams mixture-of-experts model weights from SSD, letting a 48GB model run on a 32GB laptop.
- Benchmarks on an Intel Core Ultra 9 185H show Janas-LLM beating llama.cpp's token generation speed across tested models, in one case by roughly 8-9x.
- The project released a janas-bench tool and Linux installation instructions so users can test performance on their own machines.
Crucial T700 2TB NVMe SSD — Since Janas streams model experts directly from disk instead of RAM, a fast Gen5 NVMe drive like the Crucial T700 helps keep token throughput high when running massive LLMs that exceed system memory. Its high sequential read speeds make it well suited to the kind of SSD-bound inference workload described in this article. Pairing a capable laptop with a speedy NVMe drive is key to getting the most out of local LLM streaming setups.”,
See Crucial T700 2TB NVMe SSD on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: github.com, 2026-10-12
Published there as: “Janas: Run LLMs larger than RAM by streaming experts from SSD”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.