GoKawiilPut two machines on a desk, each about $2,000. One is a tower with an NVIDIA RTX 5090: 32GB of the fastest consumer memory ever shipped, 1,792 GB/s. The other is a mini PC the size of a paperback, an AMD Ryzen AI Max+ 395 "Strix Halo" box with 128GB of soldered memory at roughly 256 GB/s. Now ask each one to run a 70-billion-parameter model.
The RTX 5090 cannot. A 70B model at a sensible 4-bit quant needs about 40GB, and 40 will not fit in 32. The little mini PC loads it without complaint, then answers at the pace of a slow reader. That paradox is the entire mini PC category in one image: these boxes can hold models that a much faster GPU cannot, and they pay for it in speed. Understanding why comes down to one idea, unified memory, and two numbers that pull in opposite directions.
We have not benchmarked these boxes ourselves. What follows synthesizes vendor specs, the inference literature, and owner-measured numbers, all linked at the end.
What "unified memory" means
In a normal desktop, the CPU has its own system RAM and the graphics card has its own separate VRAM, and data shuttles between them over the PCIe bus. A model has to fit inside the GPU's VRAM to run on the GPU, which is why a 24GB card sets a hard 24GB ceiling no matter how much system RAM you bolt on.
A unified-memory machine throws out that split. The CPU, the integrated GPU, and the NPU all share one single pool of soldered LPDDR5X memory. There is no separate VRAM, so almost the whole pool can be handed to a model. Buy the 128GB configuration and you have something close to 128GB of "VRAM" for a model to live in, for around the price of one mid-range graphics card. Apple has built Macs this way for years; AMD's Strix Halo, NVIDIA's DGX Spark, Intel's Core Ultra, and Qualcomm's Snapdragon X all now do the same. That is why mini PCs suddenly entered the local-LLM conversation at all. Capacity, cheaply.
The two numbers that decide everything
A machine's fitness for local LLMs comes down to two specs that people constantly confuse:
Capacity (how many GB of memory): decides whether the model loads at all. This is where unified-memory mini PCs win.
(how many GB of memory): decides whether the model loads at all. This is where unified-memory mini PCs win. Memory bandwidth (how many GB per second): decides how fast it generates text once loaded. This is where they lose to real GPUs, badly.
... continue reading