GoKawiilLarge Language Models (LLMs) demand immense amounts of memory, and the more people use them, the more memory is required. Memory makers responded by accelerating plans to build new memory fabs, with a focus on high-bandwidth memory (HBM) and DRAM, the first of which is scheduled to start production in 2027. But the demand for memory may also provide an opportunity for new ideas to find footing.
One of these is a tricked-out version of the kind of memory that lives in an SD card or a thumb drive—High Bandwidth Flash (HBF). It essentially takes the ideas that made HBM successful—stacking multiple chips to increase capacity and bandwidth—and applies them to the NAND flash memory commonly used for data storage in SD cards, thumb drives, and smartphones, among many other devices.
“People ask, ‘How in the world does this make a grain of sense? Flash is enormously slow,’” says Jim Handy, general director at semiconductor market research firm Objective Analysis. He explains that while NAND flash is generally lacking in bandwidth, HBF will help alleviate that concern. “[Flash] is atrociously slow for writes, but for reads, it can be coaxed to go pretty fast. And High Bandwidth Flash is going to be coaxed to do that.”
What is High Bandwidth Flash?
NAND flash stores data as a trapped electric charge in arrays of floating-gate transistors, organized into blocks and pages rather than individually addressable bytes. It’s non-volatile, too, which means data persists without power.
These traits make flash a good choice for long-term storage. It can store more bytes in the same area than DRAM, and it doesn’t require power-hungry capacitors that need constant refreshing to hold their charge. But the mechanisms that make flash dense and non-volatile also make it slow to write to, as pushing charge into and out of an insulated gate takes longer than charging a capacitor.
The latest flash interface standard can support memory bandwidth up to 4.8GB/s per die. That’s not bad for many situations, and NAND is widely used in high-performance long-term storage, such as solid state drives. However, DDR5 provides bandwidth up to 70.4GB/s per DIMM (excluding overclocked memory), and HBM4E can reach up to 3.6TB/s per stack—a roughly 750-fold bandwidth advantage for HBM4E over flash.
Hoshik Kim, senior vice president of memory systems research at SK Hynix, says HBF improves bandwidth with packaging techniques similar to HBM. “By applying advanced 3D packaging and vertical stacking techniques to NAND flash, HBF can deliver vastly higher bandwidth than standard NVMe storage,” he says. Much as HBM stacks DRAM, HBF stacks NAND flash dies to create a memory-dense chip.
HBF is at least a year away from shipping, but flash memory manufacturer Sandisk has published fact sheets for its anticipated first-generation product. The company expects HBF to stack up to 16 NAND flash chips for a total capacity of up to 512GB per stack. It also projects memory read bandwidth up to 1.6TB/s. Sandisk’s HBF roadmap also projects a second and third generation with expected read bandwidth of 2TB/s and 3.2TB/s, respectively.
What is the purpose of HBF?
... continue reading