GoKawiilHot Chips 2026 tutorial details how High Bandwidth Flash could reshape ML memory
At Hot Chips 2026, researchers Anurag Agarwal and Radhakrishna Giduthuri presented a tutorial on High Bandwidth Flash (HBF), a package-integrated flash memory technology built like HBM but functioning more like an SSD. Since no HBF hardware exists yet, the talk relied on simulations and projections, examining how software—especially inference frameworks like vLLM—would need to be redesigned to exploit HBF's large capacity despite its coarse-grained, storage-like access pattern.