Even if a GPU in a data center should only require 700 watts to run a large language model, it may realistically need 1,700 watts because of inefficiencies in how electricity reaches it. That’s a problem Peng Zou and his team at startup PowerLattice say they have solved by miniaturizing and repackaging high-voltage regulators.
The company claims that its new chiplets deliver up to a 50 percent reduction in power consumption and twice performance per watt by sizing down the voltage conversion process and moving it significantly closer to processors.
Shrinking and Moving Power Delivery
Traditional systems deliver power to AI chips by converting AC power from the grid into DC power, which then gets transformed again into low-voltage (around one volt) DC, usable by the GPU. With that voltage drop, current must increase to conserve power.
This exchange happens near the processor, but the current still travels a meaningful distance in its low-voltage state. A high current traveling any distance is bad news, because the system loses power in the form of heat proportional to the current squared. “The closer you get to the processor, the less distance that the high current has to travel, and thus we can reduce the power loss,” says Hanh-Phuc Le, who researches power electronics at the University California, San Diego and has no connection to Power Lattice.
Given the ever-growing power consumption of AI data centers, “this has almost become a show-stopping issue today,” Power Lattice’s Zou says.
Zou thinks he and his colleagues have found a way to avoid this huge loss of power. Instead of dropping the voltage a few centimeters away from the processor, they figured out how to do it millimeters away, within the processor’s package. PowerLattice designed tiny power delivery chiplets—shrinking inductors, voltage control circuits, and software-programmable logic into an IC about twice the size of a pencil eraser. The chiplets sit under the processor’s package substrate, to which they’re connected.
One challenge the minds at PowerLattice faced was how to make inductors smaller without altering their capabilities. Inductors temporarily store energy and then release it smoothly, helping regulators maintain steady outputs. Their physical size directly influences how much energy they can manage, so shrinking them weakens their effect.
The startup countered this issue by building their inductors from a specialized magnetic alloy that “enables us to run the inductor very efficiently at high frequency,” Zou says. “We can operate at a hundred times higher frequency than the traditional solution.” At higher operating frequencies, circuits can be designed to use an inductor with a much lower inductance, meaning the component itself can be made with less physical material. The alloy is unique because it maintains better magnetic properties than comparable materials at these high frequencies.
The resulting chiplets are less than 1/20th the area of today’s voltage regulators, Zou says. And each is only 100 micrometers thick, around the thickness of a strand of hair. Being so tiny allows the chiplets to fit as close as possible to the processor, and the space savings provide valuable real estate to other components.
... continue reading