GoKawiilOpen-source project squeezes multi-billion-parameter LLMs onto a budget Android TV box
A developer effort called 'smolmaxxing' has optimized large language models to run on a low-end Xiaomi TV Box S with an Amlogic S7D chip and roughly 1 GB of usable RAM. Using custom kernels, expert-paging from flash storage and memory tricks, the project reports running models ranging from tiny 260K-parameter nets up to a 16-billion-parameter mixture-of-experts model, achieving several tokens per second in real chat use.