GoKawiilslotstream
Run Qwen3.8-Flash-Next on a Mac that cannot hold it. The model is 104 GB at 4-bit; slotstream streams it from SSD and runs it in whatever memory you give it, down to an 8.1 GB planned floor. One Swift binary with the commonly used Ollama and OpenAI chat/generate endpoints.
on a 48 GB Mac Warm decode ~12 tok/s Cold start to first token ~3 s Peak memory 32 GB (auto-sized; you can cap it) Weights on disk 104 GB
Will it run on my Mac
Disk is the gate that bites first. You need ~110 GB free, so a 512 GB Mac is the realistic minimum however much memory it has. The weights are a one-time 104 GB download: well under an hour on a fast connection, several hours on a slow one (table below).
memory expect 8 GB below the 8.1 GB floor; doctor warns that it will page 16 GB ~5 tok/s estimated 24 GB ~8 tok/s estimated 32 GB ~10 tok/s estimated 48 GB and up ~12 tok/s — and auto stops at 33 GB here, so the rest of the machine stays yours
Only the 48 GB row is measured on real hardware; the rest come from the same measured curve, and smaller Macs also have slower SSDs. Run slotstream doctor to see what your machine would get, and whether you have the disk for the weights, before downloading anything.
Install
curl -fsSL https://raw.githubusercontent.com/carloslfu/slotstream/main/install.sh | sh
Installs a prebuilt binary to ~/.slotstream/bin and puts it on your PATH. Needs Apple Silicon and macOS 14+. Re-run the same line to upgrade; uninstall with rm -rf ~/.slotstream .
... continue reading