GoKawiilOpen-source project squeezes multi-billion-parameter LLMs onto a budget Android TV box
A developer effort called 'smolmaxxing' has optimized large language models to run on a low-end Xiaomi TV Box S with an Amlogic S7D chip and roughly 1 GB of usable RAM. Using custom kernels, expert-paging from flash storage and memory tricks, the project reports running models ranging from tiny 260K-parameter nets up to a 16-billion-parameter mixture-of-experts model, achieving several tokens per second in real chat use.
GoKawiil's interpretation of the reporting above, not reported fact.
The results suggest that meaningful on-device AI inference may be achievable on cheap, widely available consumer hardware rather than requiring specialized accelerators or cloud servers, which could broaden access to private, offline AI assistants. The project's own benchmarks indicate that model efficiency, not just size, determines usability on constrained hardware, which could influence how future small-footprint models are designed and optimized for edge devices.
- The project runs LLMs from 260K to 16B parameters on a sub-$50 TV box with ~1GB usable RAM.
- Techniques like expert paging, prefetching, and lookup-table kernels boost throughput several-fold over stock performance.
- A 16B mixture-of-experts model outperforms an 8B model only because of better underlying architecture, not just size.
Xiaomi Mi Box S — This is literally the device at the center of the article's benchmarks, making it a fun, cheap platform to try running small LLMs yourself. With Android TV, an Amlogic SoC and modest RAM, it's a great testbed for tinkering with on-device inference and weight streaming tricks like the ones described.
See Xiaomi Mi Box S on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: github.com, 2026-11-12
Published there as: “I'm making LLMs work on small and old devices”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.