GoKawiilAlibaba’s Qwen 3.8 27B open-weight AI model came out a couple of weeks ago, and it immediately created a wave of hype among local AI enthusiasts thanks to its impressive intelligence benchmark results for a model of its size and capabilities.
Totaling around 17GB for four-bit quantized weights and offering built-in multimodal capabilities on top of its general aptitude, Qwen 3.8 27B immediately grabbed the attention of everybody with an RTX 5090, RTX 4090, or RTX 3090 (as well as a Radeon RX 7900 XTX, Radeon AI Pro R9700, or Arc Pro B70).
Were we on the verge of frontier-level intelligence from a four-bit quant on a single graphics card? Could everybody with a capable enough local AI setup go and cancel their Claude or ChatGPT subscriptions?
Latest Videos From Tom's Hardware Watch full video here:
The answer, of course, as with every open-weight AI model hype cycle, is more complicated than just eyeballing the size of the model weights and comparing it to your available VRAM pool. Does the card or system you're using to host the model have enough VRAM left over to provide useful amounts of space for the model's context once everything is running? Do your host system and LLM inference engine deliver acceptable time-to-first-token, as well as high throughput beyond just bench-racing from an empty context window?
It's one thing if you just want to chat with a model and see what happens; it's another entirely if you want to put it to work, especially as impatient agents take the limits of human perception out of the picture.
We wanted to see what hardware and software stack Qwen 3.8 27B really wants in order to deliver solid performance, so we ran it on systems ranging from a desktop PC with discrete GPUs to systems with unified memory architectures like the DGX Spark, Mac Studio, and Ryzen AI Halo.
Our discrete GPU AI testbed includes the following components:
Swipe to scroll horizontally Tom’s Hardware Local AI Testbed Row 0 - Cell 1 CPU Ryzen 7 9800X3D Memory 64GB (4x16GB) DDR5-5200 Motherboard Asus TUF Gaming X670E-Plus Wifi SSD Corsair MP600 Pro XT 4TB Power supply MSI MPG Ai1600TS Operating system Ubuntu 26.04 LTS
Where it was possible to do so, we tested performance with Qwen 3.8 27B’s built-in multi-token prediction capabilities both enabled and disabled. Not all of the model runners we tested were able to support MTP within the amount of VRAM available to us on some of our platforms. We note where MTP was and wasn’t possible in our analysis of each platform, as well as in our charts.
... continue reading