GoKawiilOnly a little while back, Phoronix got the chance to test-drive one of Nvidia's upcoming Arm-based Vera CPUs. In certain approved workloads, the chip put up an impressive showing, nipping at the heels of its Xeon and Epyc x86 competitors. In specific single-threaded scenarios, Vera "absolutely dusted the competition" (our words). But AMD had some things to say about the Phoronix test, firing back with its own metrics of a 3.3x performance gain over Vera for the projected output of a 100 kW rack of its hardware.
And Nvidia is already thinking about this future. It revealed that its next-gen Rigel Arm v9.2 CPU core, shipping as part of its Rosa CPU, will deliver even higher per-core performance than Vera's Olympus core within the same silicon footprint via "better instruction delivery," more L2 cache, and better memory handling.
Now, Nvidia is reasserting Vera's advantage for AI work by describing it with a new product category: a "max single-threaded CPU at scale" rather than a parallel-processing beast. Instead of simply maximizing the core count per socket, Nvidia says Vera's monolithic 88-core design is meant to provide strong performance per core under load, enough memory bandwidth per core to keep active cores supplied with data, and predictable latency.
Latest Videos From Watch full video here:
Nvidia describes AI inference workloads as being bound by single-thread speed. For example, a reasoning AI will run the model for one step, and will run the model again as many times as it takes until the answer is generated. Since each step needs the output from the previous one, no amount of parallelism will help — the speed at which one thread can run is most important. The situation is similar in agentic workloads, as agent B can't get its work started without knowing what happened with agent A.
(Image credit: Nvidia)
Vera's design, then, appears to be one aimed at both having and eating the proverbial cake: high single-thread speed with a large number of available threads. Vera is an 88-core design with SMT support for 176 total threads. And to supply each of those cores with adequate bandwidth, Nvidia says Vera talks to LPDDR5X RAM at 1.2 TB/s, and that its monolithic compute die keeps cores well fed and avoids bottlenecks thanks to 3.4 TB/s of core-to-core bandwidth. The company says the latter figure is 3x that of "any other data center CPU."
There are many ways to measure inter-core bandwidth, so direct comparisons are tricky at best, but given the bespoke design of Vera for AI inference tasks, the claim is at least plausible.
The company's latest blog post about the new silicon reiterates this point, claiming its new silicon delivers 1.8x higher performance versus its x86 competition in "loaded CPU workloads that represent agentic execution," 1.5x higher perf in coding workflows, and 3x faster work in database analytics.
Stay On the Cutting Edge: Get the Tom's Hardware Newsletter Get Tom's Hardware's best news and in-depth reviews, straight to your inbox. Contact me with news and offers from other Future brands Receive email from us on behalf of our trusted partners or sponsors
... continue reading