HBF Opens the TB Era for GPU Memory, Surpassing HBM Through NAND Stacking

GPU memory capacity, which was limited to a few GB with conventional HBM (High Bandwidth Memory), could now be boosted to the TB class using a new approach that applies storage technology. High-Bandwidth Flash (HBF) developed by US-based Sandisk and South Korea’s SK Hynix packages NAND flash using the same 3D stacking technology as HBM, combining SSD-like large capacity with HBM-class bandwidth. According to a report by Tobias Mann of The Register, Sandisk’s first-generation HBF achieves a read bandwidth of 1.6TB/s and a capacity of 512GB. However, the inherent walls of NAND write endurance and access latency remain significant, making it seem more realistic for HBF to be adopted in limited use cases rather than as a complete replacement for HBM.

Background: HBM Capacity Limits

In the current AI accelerator market, HBM has become the standard technology directly integrated into GPUs and dedicated chips. HBM3e offers up to 24GB per stack and 2.5TB/s bandwidth with 12 layers. HBM4, whose mass production will ramp up in 2026, will reach 64GB per stack and over 2.5TB/s bandwidth with 16 layers. However, large language models operated by cloud providers have hundreds of billions of parameters and do not fit into a single GPU, making model parallelism and tensor parallelism across multiple GPUs essential. HBM capacity is limited to a few GB, and beyond that, inter-GPU connection bandwidth and latency become bottlenecks.

Technical Features of HBF

HBF adopts a package architecture very similar to HBM. Sandisk’s first-generation HBF stacks 16 layers of NAND flash and achieves a read bandwidth of 1.6TB/s. This exceeds HBM3e (approximately 1.2TB/s) but falls short of HBM4 (2.5TB/s). However, in terms of capacity, each module offers up to 512GB, which is more than 14 times the capacity of HBM4’s 36GB (16 layers).

Sandisk has announced plans to increase bandwidth to 2TB/s and further to 3.2TB/s in future generations. Because HBF modules conform to the same package dimensions as HBM, they can be directly integrated onto GPU dies using advanced packaging technologies such as TSMC’s CoWoS and Intel’s EMIB/Foveros. The ability to adopt HBF without major changes to existing GPU/accelerator designs is a significant advantage for widespread adoption.

Power Efficiency and Cost Advantage

Sandisk claims that HBF’s power consumption and price will be equal to or lower than those of HBM. Since NAND flash has a much lower cost per bit than DRAM, significantly larger capacity can be achieved with the same budget. In workloads that are read-intensive with infrequent writes, such as inference, HBF may outperform HBM in cost-performance.

Fundamental Challenges: NAND Weaknesses

However, as noted in The Register article, HBF cannot completely replace HBM. The biggest barriers are NAND flash’s write endurance and access latency. DRAM access latency is tens of nanoseconds, while NAND flash is tens of microseconds—about 1,000 times slower. Additionally, NAND has a limited number of program/erase cycles per cell, especially for multi-level cells like QLC and PLC where endurance is restricted to a few thousand to tens of thousands of cycles.

In AI model training executed by GPUs, frequent weight updates occur via backpropagation. Using HBF for training would cause the compute units to wait due to insufficient write throughput, and significantly shorten NAND lifespan. Sandisk itself appears to envision inference as the primary use case for HBF.

Furthermore, while HBM bandwidth is equally fast in both read and write directions, HBF is optimized for reads, and write performance is much lower. Even in Sandisk’s published specifications, only read bandwidth is shown; details on write bandwidth and latency are not disclosed.

Realistic Use Cases

HBF is suitable for workloads that are mainly read-intensive with rare writes, such as inference-dedicated accelerators, memory servers that statically hold model weights, or cache layers for large-scale retrieval-augmented generation (RAG) systems. It can also be used as high-speed storage for saving checkpoints during training.

Currently, Sandisk and SK Hynix are pursuing the standardization of HBF and establishing mass production technology, targeting product launches after 2027. While HBF holds the potential to break through the capacity wall faced by HBM, the key to practical success will be how well NAND’s physical limitations can be mitigated at the system level.

Editorial Opinion

HBF deserves high praise for presenting a practical solution to the serious AI industry problem of HBM capacity shortage, applying insights from storage technology. In the short term, in inference-specific GPUs and AI accelerators, products equipped with HBF could bring significant cost reductions and increased model sizes compared to conventional approaches. In particular, if ultra-large models can be accommodated in a single GPU, inference server configurations will be simplified, improving capital efficiency for cloud providers. From a long-term perspective, it will be interesting to see whether HBF is positioned as a complementary technology rather than a replacement for HBM, or whether improvements in NAND—such as ferroelectric NAND or MRAM-based non-volatile memory—will enhance write endurance and expand HBF into training applications. Furthermore, if HBF adoption progresses, the era of GPU memory capacity reaching several TB will arrive, and memory management strategies on the software side will also change significantly. However, whether HBF truly becomes a viable product can only be judged after specific figures for write performance and endurance are disclosed.

References

• “GPUs could explode to multiple TB with new storage-inspired memory tech”, by Tobias Mann — The Register, 2026-07-30T18:33:00.000Z (ARR)
• Source URL: https://www.theregister.com/storage/2026/07/30/gpus-could-explode-to-multiple-tb-with-new-storage-inspired-memory-tech/5281363