GoKawiilSwiftlet
Run 35B and 80B Qwen models on ordinary Apple devices, including iPhones.
Swiftlet is a Swift + Metal runtime for the Qwen3-Next and Qwen3.5/3.6 MoE hybrid model family. It keeps only the small dense core of a model resident in memory and streams the routed Mixture-of-Experts weights from storage on demand. The result:
Model Disk Peak RAM Decode speed (M5 Mac) Qwen3.6-35B-A3B, 4-bit 18 GB 2.6 GB 7 to 11 tok/s Qwen3-Next-80B-A3B, 4-bit 42 GB 4.3 GB 4.5 to 5 tok/s
The 35B also runs on an iPhone 17 in about 2.5 GB of RAM, at about 1 tok/s today. As far as we know, that is the first time a model of this class has run natively on a phone.
Status: working end to end. Both models generate correct, validated output. The current focus is kernel speed (the decode loop is dispatch bound, not IO bound, so there is clear headroom). One expectation to set honestly: only about 3B parameters are active per token, so these models chat and write like large models but recall facts like small ones.
Quick start: try it on a Mac
git clone https://github.com/leonickson1/Swiftlet.git && cd Swiftlet swift build -c release # Download the 35B container from Hugging Face (resumable): .build/release/swiftlet-repack \ --from-hf Leonickson/Qwen3.6-35B-A3B-qpack \ --output ~ /models/qwen3.6-35b.qpack # Or the 80B (42 GB on disk, still only ~4.3 GB of RAM): .build/release/swiftlet-repack \ --from-hf Leonickson/Qwen3-Next-80B-A3B-qpack \ --output ~ /models/qwen3-next-80b.qpack # Chat (applies the model chat template, disables the reasoning block, # keeps conversation state so follow-ups prefill only the new turn): .build/release/swiftlet chat ~ /models/qwen3.6-35b.qpack \ " Who wrote One Hundred Years of Solitude? " " What language did he write it in? " # One-shot generation with stats: .build/release/swiftlet generate ~ /models/qwen3.6-35b.qpack \ --gpu --chat --prompt " Explain expert streaming in one paragraph. " # OpenAI-compatible server (loopback only): .build/release/swiftlet-server --model ~ /models/qwen3.6-35b.qpack --port 8080
The same command also repacks raw MLX checkpoints ( --from-hf mlx-community/... or --source /path/to/checkpoint ).
Requirements: Apple Silicon, macOS 14+ or iOS 17+, free SSD space for the container (18 GB for the 35B, 42 GB for the 80B).
... continue reading