GoKawiilOpen-source llama-modchip runs 284B-parameter DeepSeek model on 64GB CPU-only machine
A project called llama-modchip, released on GitHub, claims to compress the 284.33B-parameter DeepSeek-V4-Flash mixture-of-experts model from roughly 77GB down to about 57-59GB of memory, letting it run on a 64GB CPU-only Linux machine without a GPU. The developers report prefill speeds near 44 tokens per second and CPU decode speeds of roughly 9-15 tokens per second, achieved through custom quantization, cache-tiling and pruning techniques, plus a separate mode running a 30B Nemotron model at much higher speeds on CPU hardware.