At Hot Chips, Micron disclosed a key data point: producing 1GB of HBM (High Bandwidth Memory, the dedicated memory for AI chips) consumes roughly 3x the wafer area of standard DDR5. We believe this directly explains why AI compute costs have not come down—HBM is steadily eating into global memory capacity.
What this is
Micron's technical disclosure centered on a simple comparison: HBM4 has 256 memory banks per die, while DDR5 has only 32. Stack on additional data paths, power delivery modules, and TSVs (Through-Silicon Vias, the vertical wires connecting stacked chips), and total die area naturally balloons.
Translated to actual products: a single Nvidia B100 carries 144GB of HBM, effectively displacing 432GB of standard DDR5 capacity. All three major memory makers—Micron, Samsung, and SK Hynix—have collectively pivoted to HBM, causing global memory output, measured in GB produced, to shrink by two-thirds.
More critically, a Micron Fellow added that next-generation HBM will not improve this ratio.
Industry view
The mainstream read: this is the underlying explanation for current HBM shortages, DRAM price hikes, and persistently high AI server costs. Even if all new 2025 wafer capacity were redirected back to DRAM, supply tightness won't ease anytime soon.
But we need to hear the other side: this is Micron's own narrative. All three OEMs are simultaneously pushing HBM for higher margins, passing 3x wafer costs to AI customers. The flip side of DRAM price hikes is record vendor profits. Moreover, the analysis assumes new capacity will "trickle back" to standard memory—but reality is that new fabs are designed from the ground up for HBM. Supply tightness may be structural, not cyclical.
Impact on regular people
For enterprise IT: Plan self-built AI cluster hardware budgets around sustained elevated pricing. Applications relying on cloud inference need to budget for API price hikes.
For individual careers: Internal corporate AI project timelines may stretch; the "six-month payback" assumption needs to be reworked.
For the consumer market: Memory price hikes have already passed through to PCs and phones. The AI PC "upgrade cycle dividend" may be eaten by hardware costs.