GoKawiilReport details GPU, CPU and NPU benchmarks for running local AI models
New testing data compares how different hardware handles local large language models, measuring output in tokens per second. Nvidia's RTX 5090 generated 205 tokens per second on gpt-oss-20b, Apple's M4 chip produced about 24 tokens per second on a 7B model, and Nvidia's DGX Spark ranged from 38 tokens per second on Llama 3.1 8B down to 4.42 on its 70B version. Separate hands-on testing on a 2024 Asus Zenbook Duo showed Intel's OpenVINO toolkit offloading work to an NPU kept the laptop responsive, versus heavy CPU and memory strain when running the same model through Ollama.