GoKawiilTurboFieldfare
Gemma 4 26B-A4B inference in about 2 GB of RAM
A custom Swift + Metal runtime for any Apple Silicon Mac, even the 8 GB ones.
Quick start · Local server · Benchmarks · Contribute results · How it works · Experiments · References
Memory got expensive. So I gave a 26-billion-parameter model a ~2 GB budget.
TurboFieldfare runs the instruction-tuned Gemma 4 26B-A4B without loading the entire 14.3 GB model into memory. It keeps the shared 1.35 GB core and FP16 KV cache in memory, then streams only the experts needed for each token from SSD. This is what lets the model run on Macs with 8 GB of RAM.
The runtime, streaming installer, CLI, and native Mac app are written in Swift and Metal. TurboFieldfare is model-specific rather than a wrapper around MLX or llama.cpp. The curated experiment record summarizes 103 measured results across kernels, caching, I/O, prefill, and decode.
Try it
git clone https://github.com/drumih/turbo-fieldfare.git cd turbo-fieldfare swift build -c release .build/release/TurboFieldfareMac
On the first run, Swift Package Manager downloads and builds the Swift packages required by the tokenizer. The complete release build includes the foreground Mac app and its sibling decode-service executable.
... continue reading