GoKawiil____ _ _ __ _ | _ \ ___| | |_ __ _ / _(_)_ __ | | | |/ _ \ | __/ _` | |_| | '_ \ | |_| | __/ | || (_| | _| | | | | |____/ \___|_|\__\__,_|_| |_|_| |_| An experiment in running Kimi K3 (2.8T parameters) on one Apple Silicon Mac Deltafin is a small research project that runs a Mixture-of-Experts model far larger than the machine it sits on. It is not fast — about 16 seconds per token on our M1 Max — but it is exact, reproducible, and it works on a 64 GB laptop. Newer chips and more RAM make it faster automatically.
Install
Three commands, then you're generating. The only real decision is step 3.
# 1. environment (Python 3.12+, and Xcode CLT for clang) python3 -m venv venv ./venv/bin/pip install torch numpy safetensors tiktoken ml_dtypes blobfile \ " transformers==4.56.2 " einops tokenizers # 2. build the fused MXFP4 kernel clang -O3 -mcpu=native -shared -DNO_MAIN -o tools/libmxfp4gemv.dylib tools/fused_gemv.c # 3. download the model (see the two modes below) ./venv/bin/python tools/setup_k3.py --full
The two modes
--full (recommended) --stream Disk needed ~1.7 TB ~215 GB Download time 5–10 hours, resumable ~30 minutes Speed afterwards ~60–76 s/token, every prompt ~3+ min/token for anything not already cached Network at inference none constant
Every token reads 16 experts × 92 layers = 25.8 GB of expert data. From local disk that's about 4 seconds; over the network it's minutes. That single fact is the whole difference between the two columns.
Run setup_k3.py with no flag and it picks --full when the disk allows, otherwise falls back to streaming and tells you exactly how much space you'd need to free.
Starting with streaming and upgrading later
Streaming is a fine way to try Deltafin without committing 1.7 TB. Whenever you want the speed, one command finishes the job — no reinstall, no reconfiguration, and it picks up whatever is already cached:
... continue reading