HBM

1. Overview

High Bandwidth Memory (HBM) is a computer memory interface technology for 3D-stacked synchronous dynamic random-access memory (SDRAM), jointly developed by Samsung, AMD, and SK Hynix. In October 2013, HBM was adopted as an industry standard (JESD235) by JEDEC. The technology vertically stacks multiple DRAM dies, using Through-Silicon Vias (TSVs) and microbumps to achieve inter-layer interconnection, and is then tightly coupled with a GPU or accelerator via a silicon interposer, thereby delivering data transfer bandwidth tens of times greater than traditional DDR/GDDR solutions with far smaller volume and lower power consumption.

The first commercial HBM chip was manufactured by SK Hynix in 2013, and the first product equipped with HBM was the AMD Fiji series GPU launched in 2015. Since then, HBM has continued to iterate: HBM2 (standardized in 2016), HBM2E (2019), HBM3 (2022), HBM3E (2023), and in April 2025, JEDEC officially released the HBM4 standard, doubling the interface width to 2048-bit and achieving a per-stack bandwidth of 2 TB/s.

The global HBM market size in 2025 is approximately $38 billion, and it is expected to grow to $58 billion in 2026. In this market, SK Hynix leads by a wide margin with a 62% market share, Micron is second with 21%, while Samsung, hindered by HBM3E certification issues, saw its share plummet from 41% in Q2 2024 to 17% in the same period of 2025. The growth driver for HBM comes almost entirely from the demand for AI infrastructure—one HBM stack’s wafer capacity can consume the equivalent of three DDR5 wafers, leading to ongoing tight supply of general-purpose DRAM.

2. Technical Architecture

2.1 3D Stacking Structure

The fundamental difference between HBM and traditional 2D planar packaged memory (DDR/GDDR) lies in its vertically stacked architecture. A typical HBM stack includes the following layers from bottom to top:

• GPU/Accelerator Die: Located at the bottom of the package, performing computation functions.
• Silicon Interposer: A silicon substrate that hosts the GPU and HBM stacks, providing high-density interconnect wiring. For large packages of 55mm to 65mm, the interposer is currently the only mature solution.
• Base Logic Die: Located at the very bottom of the DRAM stack, integrating buffering, testing, and controller logic. TSMC is the primary foundry for the base logic dies of multiple HBM manufacturers.
• DRAM Die Layers: Vertically stacked from 1 layer up to 16 layers. The number of layers determines the total capacity—HBM1 used 4-high stacking, and HBM4 supports up to 16-high.

The DRAM layers are vertically interconnected through TSVs and microbumps. TSVs are conductive channels that penetrate the entire silicon die, providing the shortest inter-layer signal path; microbumps are solder ball connection points that form both electrical and mechanical connections between layers.

2.2 Key Interconnect Technologies

TSV (Through-Silicon Via): Tiny vertical conductive channels penetrating the silicon die, with the pitch gradually shrinking from about 40µm in early HBM to an expected ~10µm in HBM4. TSV pitch directly determines the size of the microbumps—the smaller the pitch, the smaller the bumps.

Microbumps: The current standard inter-layer connection method for HBM stacking. Before HBM4, the microbump pitch was around 40µm; HBM4 pushes this pitch close to about 10µm. JEDEC relaxed the HBM4 stack height limit from 720µm to 775µm, making it possible to continue using microbumps. This decision temporarily postpones the large-scale adoption of hybrid bonding.

Hybrid Bonding: A solder-free direct Cu-Cu bonding technology that, after eliminating microbumps, can further reduce the inter-layer spacing. Hybrid bonding can reduce energy consumption by approximately an order of magnitude. However, this technology faces significant yield and testing challenges: the bonding interface requires zero particle contamination, test probes may damage the bonding surface, and therefore special processes such as surface planarization repair are needed. HBM4 continues to use microbumps, and hybrid bonding is expected to be adopted beginning with HBM4E or HBM5.

Interposer: The silicon interposer hosts the GPU and up to 8–16 HBM stacks. Its process requirements are extremely high—the wiring density within a 55–65mm package size far exceeds what can be achieved with PCBs. The manufacturing cost of the interposer is a significant component of the overall HBM package cost.

2.3 Channel Architecture

HBM’s bandwidth advantage stems from its ultra-wide bus architecture:

• HBM1/HBM2/HBM2E: 8 channels × 128-bit per stack = 1024-bit total width.
• HBM3: The number of channels doubles to 16 channels, but the width per channel decreases to 64-bit, maintaining a total width of 1024-bit.
• HBM4: The number of channels doubles again to 32 channels × 64-bit = 2048-bit total width. This change represents the largest interface architecture transformation in HBM history, affecting the entire memory subsystem—the memory controller, PHY layer, and interposer wiring all require redesign.

The formula for calculating HBM bandwidth is: Total Bandwidth (GB/s) = Data Rate (GT/s) × Bus Width (bit) / 8. Taking HBM4 as an example: 8 GT/s × 2048-bit / 8 = 2048 GB/s (2 TB/s).

2.4 HBM-PIM (Processing-in-Memory)

Samsung’s HBM-PIM, released in 2021, embeds AI computing engines directly inside each DRAM bank, allowing data to be processed at the storage location and drastically reducing data movement. The technology claims to provide 2× system performance improvement and over 70% energy reduction without requiring any modifications to system hardware or software. HBM-PIM represents an important direction in HBM’s evolution from a purely storage device to a near-memory computing platform.

Sources: Wikipedia; Samsung HBM-PIM Press Release (2021); SemiEngineering (Jan 2026)

3. Generational Evolution

3.1 Specification Comparison Across Generations

From HBM1 to HBM4, over approximately 13 years, the per-stack bandwidth has increased from 128 GB/s to 2 TB/s, an improvement of about 16×; capacity has grown from 4 GB to 64 GB, a 16× increase; and stack depth has increased from 4 layers to 16 layers, a 4× increase.

3.2 Key Technological Milestones by Generation

HBM1 (2013): Established the fundamental architecture of 3D-stacked DRAM and TSV interconnection. SK Hynix manufactured the first chip, and the AMD Fiji GPU was the first commercial product.

HBM2 (2016): Raised the pin speed to 2.4 Gb/s and supported 8-high stacking. Samsung was the first to announce mass production in January 2016, the same month JEDEC published the standard. The NVIDIA Tesla P100 was the first GPU to adopt HBM2.

HBM2E (2019): Speed increased to 3.6 Gb/s, supported 12-high stacking, with a maximum per-stack capacity of 24 GB. Samsung’s Flashbolt (3.2 GT/s, 16 GB/stack, 410 GB/s) and SK Hynix’s HBM2E (3.6 GT/s, 460 GB/s) were introduced in 2019 respectively.

HBM3 (2022): Key architectural change: the channel architecture shifted from 8×128-bit to 16×64-bit, with the total width remaining at 1024-bit. Pin speed reached 6.4 Gb/s, with a per-stack bandwidth of 819 GB/s. SK Hynix was the first to mass-produce in June 2022, accompanying the NVIDIA H100 GPU. HBM3 is the core memory technology underpinning the current wave of large-scale AI model training.

HBM3E (2023): A speed-enhanced version built on the HBM3 architecture, with pin speed increased to 9.8 Gb/s and per-stack bandwidth reaching 1.23 TB/s. All three manufacturers—SK Hynix, Micron, and Samsung—introduced their respective HBM3E products, among which Micron was the first to achieve 9.6 Gb/s and secured design wins for NVIDIA H200/B200. SK Hynix was the first to complete mass production of 12-high (Sep 2024) and 16-high (Nov 2024) HBM3E stacks.

HBM4 (2025): The most significant architectural upgrade since the inception of HBM. The core change is the doubling of interface width to 2048-bit, with the number of channels increasing to 32. The pin speed is slightly lower than HBM3E (8 GT/s), but thanks to the doubled bus width, it still achieves 2 TB/s of bandwidth. JEDEC officially released the standard in April 2025. SK Hynix has completed development and claims its product exceeds JEDEC specifications by 25% (10 GT/s). See Section 7 for details.

3.3 HBM4 Key Specification Details

The official HBM4 specification includes the following core parameters:

• Interface Width: 2048-bit (twice that of HBM3)
• Data Rate: Up to 8 Gb/s/pin
• Per-Stack Bandwidth: 2 TB/s (an 8-stack GPU can exceed 13 TB/s)
• Channel Configuration: 32 channels × 64-bit (or 64 32-bit pseudo-channels)
• Stack Depth: 4 to 16 layers
• DRAM Die Density: 24 Gb or 32 Gb
• Per-Stack Capacity: Up to 64 GB
• Backward Compatibility: Compatible with HBM3 controllers
• Bonding Method: Microbumps; hybrid bonding postponed to later generations

Sources: Wikipedia; JEDEC HBM4 Standard (April 2025); Rambus HBM4 Memory Controller

4. Industry Competitive Landscape

4.1 Triopoly and Dramatic Market Share Shifts

The global HBM market is monopolized by three DRAM giants: SK Hynix, Samsung Electronics, and Micron Technology. Over the past two years, the market landscape has undergone dramatic changes.

HBM Market Share Distribution in Q2 2025:

• SK Hynix: 62% (Q2 2024: ~51%)
• Micron: 21% (Q2 2024: ~5%)
• Samsung: 17% (Q2 2024: ~41%)

Samsung’s share plummeted from 41% to 17% within one year, while SK Hynix and Micron gained significant growth. The direct cause of this shift was Samsung’s HBM3E failing to pass NVIDIA’s quality certification tests, causing it to lose a large number of orders in the prime AI memory market. This landscape change also triggered a broader DRAM market reshuffle—in Q1 2025, SK Hynix surpassed Samsung for the first time to become the world’s largest DRAM manufacturer (36% vs. 34%).

4.2 SK Hynix: Market Leader

SK Hynix has established a series of “first” records in the HBM field:

• World’s first to mass-produce HBM chips (2013)
• World’s first to mass-produce HBM3 and supply for NVIDIA H100 (Jun 2022)
• World’s first to mass-produce HBM3E 12-high stack (Sep 2024)
• World’s first to mass-produce HBM3E 16-high stack (Nov 2024)
• World’s first to complete HBM4 development, with performance exceeding the JEDEC standard by 25% (10 GT/s vs. 8 GT/s)

SK Hynix has become NVIDIA’s preferred HBM supplier, a relationship that directly drives its market leadership. The company is massively expanding production capacity—monthly capacity is moving from 10,000 wafers (2024) towards over 70,000 wafers (2025–2026). HBM4 mass production is planned to begin in the first half of 2026.

4.3 Samsung: From Leader to Chaser

Samsung’s position reversal in the HBM field is arguably one of the most striking events in the memory market in recent years:

• Q2 2024: Market share ~41%, roughly on par with SK Hynix
• Key Turning Point: HBM3E failed to pass NVIDIA’s stringent certification testing
• Impact: Samsung’s HBM sales relied primarily on the older generation HBM3, while competitors had fully shifted to HBM3E shipments
• Market share fell sharply to: 17% (Q2 2025)

Samsung’s HBM4 strategy is accelerating. In Q3 2025, Samsung began shipping large quantities of HBM4 samples to NVIDIA for early certification. Concurrently, according to industry reports, Samsung will become the primary HBM4 supplier for AMD’s MI450 accelerator. Samsung’s HBM4 mass production target is set for the first half of 2026. Analysts expect that as its HBM3E products pass certification and HBM4 enters large-scale supply, Samsung’s market position will be partially restored in 2026.

4.4 Micron: The Resilient Chaser

Micron is the smallest in production scale among the three major players, but its HBM strategy execution has been highly precise:

• Technology Breakthrough: HBM3E was the first to reach a pin speed of 9.6 Gb/s (surpassing SK Hynix)
• NVIDIA Certification: Successfully secured design wins for H200 and Blackwell B200
• Market Share Growth: Increased from ~5% (Q2 2024) to 21% (Q2 2025), a gain of more than 300%
• HBM4 Progress: Began shipping HBM4 samples to customers in June 2025, offering 36 GB 12-high stacks; in Q4 2025, announced samples achieving speeds of 11+ Gb/s, corresponding to a per-stack bandwidth of over 2.8 TB/s
• Mass Production Timing: Targeting 2026

Micron’s HBM4E products also particularly emphasize the R&D optimization of hybrid bonding technology—the company has filed over 80 patent applications related to hybrid bonding for high-bandwidth memory.

Sources: Introl Blog (Feb 2026); PatSnap HBM Landscape (Mar 2026); Astute Group (2025); Blocks & Files (May 2025)

5. Application Scenarios and Market

5.1 HBM Configurations in AI Accelerators

HBM is the core storage component in current AI training and inference infrastructure. Below is a comparison of memory configurations for mainstream AI GPUs:

Noteworthy data points include:

• The H200 increased memory capacity by 76% (80 GB to 141 GB) and bandwidth by 43% compared to the H100, yet used the same Hopper architecture core—the performance improvement came almost entirely from the HBM3E memory subsystem upgrade.
• The AMD MI300X, with 192 GB HBM3 and 5.3 TB/s bandwidth, surpasses the contemporary H100 by more than a factor of two in both memory capacity and bandwidth. This allows the MI300X to run large-scale models like Llama 3 405B within a single node, whereas the H100 requires multiple nodes.
• The NVIDIA Rubin (2026) interposer area reaches 2,194 mm², with an estimated power consumption of 2,200W and a total bandwidth of up to 32 TB/s. Memory configuration grows from the A100’s 80 GB HBM2E to the Rubin Ultra’s 1,024 GB HBM4E, an increase of approximately 12.8× over six years.

5.2 Memory Demand Driven by Large Language Models

The training and inference of large language models place extremely high demands on memory bandwidth and capacity. During large model inference, the Attention mechanism requires accessing the complete Key-Value Cache on a per-token basis—memory bandwidth directly determines the throughput in tokens-per-second. Training workloads face different constraints: model parameters, gradients, optimizer states, and activations all compete for limited capacity resources.

HBM capacity has grown from 80 GB (A100/H100) to 192 GB (B200/MI300X) and further to 384+ GB (Rubin/MI400), directly serving the evolution demands of models scaling from tens of billions to trillions, and even tens of trillions, of parameters.

5.3 Market Size and Growth

• 2025 Global HBM Market: ~$38 billion
• 2026 Global HBM Market (Forecast): ~$58 billion
• Growth Drivers: AI infrastructure investment, growth in large model parameters, emergence of multimodal models
• Pricing Impact: HBM remains in short supply. Since the beginning of 2025, DRAM prices have cumulatively increased by over 200%. Micron points out that the wafer capacity conversion ratio between HBM and DDR5 is approximately 1:3—meaning that for every HBM wafer produced, the capacity consumed is equivalent to sacrificing three DDR5 wafers. SK Hynix, Samsung, and Micron’s HBM production capacity through 2026 is essentially sold out.

Sources: GPU Advisor Benchmarks (2026); SemiAnalysis Newsletter; Blocks & Files (May 2025)

6. Technical Challenges and Bottlenecks

6.1 Memory Wall

The “Memory Wall” is the fundamental constraint faced by AI accelerators—the growth rate of computing power continuously outpaces the growth rate of memory bandwidth. Each HBM generation improves bandwidth by about 1.5 to 2×, while each GPU generation improves computing power (measured in TFLOPS) by about 2 to 3×. This gap directly dictates the upper limit of LLM inference throughput. For memory-bound inference workloads, memory bandwidth is a more critical bottleneck parameter than peak compute power.

6.2 Thermal Management

The 3D stacking structure introduces inherent thermal management challenges. The structural thermal resistance between vertically stacked DRAM layers intensifies as the number of layers increases. Intermediate DRAM layers lie along the longest heat dissipation path and are prone to forming hotspots. The power consumption of a single HBM4 stack is expected to reach 80W; a GPU equipped with 8 HBM4 stacks would have to handle over 640W of thermal load from the memory subsystem alone. Package-level thermal management thus becomes one of the key bottlenecks for further HBM stacking.

6.3 Yield and Testing

The high stacking layer count of HBM amplifies yield challenges. A single unrecoverable DRAM die defect in a 16-high stack can result in the entire stack being scrapped. The Known-Good Die (KGD) strategy is critical—testing each layer individually before stacking. However, the introduction of hybrid bonding technology makes the testing process extremely difficult: the bonding interface must maintain zero particle contamination before bonding, yet the test probe itself is a particle source. Foundries like UMC are developing intermediate testing processes that include surface planarization repair.

6.4 Advanced Packaging Costs

The manufacturing cost of silicon interposers for package sizes of 55–65mm is exceptionally high. HBM4 pushes the microbump pitch to about 10µm, requiring a new generation of packaging equipment. The equipment investment for hybrid bonding (TCB, direct Cu-Cu bonding machines) is even higher and necessitates entirely new process flows. These costs will ultimately be reflected in the pricing of HBM products.

6.5 Capacity Squeeze Effect

As previously noted, HBM consumes a tremendous amount of DRAM production line capacity. Every single HBM wafer is equivalent to occupying the production capacity of three DDR5 wafers. This means that the explosive growth in HBM demand directly leads to tight supply and rising prices of general-purpose DRAM (DDR5, LPDDR), creating a structural contradiction between AI memory demand and traditional PC/server memory demand.

Sources: SemiEngineering (Jan 2026); PatSnap Thermal Analysis (Apr 2026); Georgia Tech / SK Hynix 3D Stacked HBM Paper

7. Future Trend Outlook

7.1 HBM4E (Target 2027)

HBM4E is the enhanced version of HBM4, with JEDEC planning to release it around 2027. Key target specifications:

• Data rate increased to 10 GT/s
• Per-stack bandwidth reaching 2.5 TB/s
• Per-stack power consumption cap: 80W
• Bonding scheme: Hybrid bonding may be introduced for the first time in 18-high or 20-high stacks

7.2 Roadmap for Hybrid Bonding Introduction

The adoption timeline for hybrid bonding (Cu-Cu direct bonding) has a relatively clear outline:

Hybrid bonding can eliminate solder bumps, bringing the inter-layer spacing close to zero and significantly reducing impedance and power consumption (roughly an order of magnitude lower compared to microbumps). However, the cost is high—it requires entirely new production lines and testing equipment, and yield control is more complex. Equipment manufacturers such as TSMC, Applied Materials, and BESI are driving collaborative optimization of bonding equipment and processes.

7.3 Next-Generation AI Accelerator Roadmap

Both NVIDIA and AMD’s next-generation AI accelerators will adopt HBM4:

NVIDIA Rubin (HBM4):

• Release Date: Second half of 2026
• Configuration: 8 HBM4 stacks
• Memory Capacity: 288–384 GB (varies by stack configuration)
• Total Bandwidth: 16–32 TB/s
• Package Size: 2,194 mm² interposer
• Power Consumption: ~2,200W

AMD MI400 (HBM4):

• Release Date: 2026
• First Model: MI430X
• Memory Capacity: 432 GB
• Total Bandwidth: 19.6 TB/s

NVIDIA Rubin Ultra (HBM4E):

• Configuration: 16 HBM4E stacks
• Memory Capacity: 1,024 GB
• Target Timeframe: 2027

7.4 Long-Term Roadmap

According to industry analysis, the roadmap from HBM5 to HBM8 is beginning to take shape, targeting a system-level bandwidth of 64 TB/s and beyond. Key technology paths in this roadmap include:

• Hybrid Bonding: Becoming the mainstream stacking solution starting with HBM5
• Processing-in-Memory (PIM): The subsequent evolution of Samsung’s HBM-PIM, integrating more computing logic into the DRAM
• 3D DRAM Architecture: Yole Group predicts that customized HBM and novel 3D DRAM architectures will drive the next generation of memory innovation
• CXL Integration: The combination of HBM with CXL interconnect to expand memory pooling capabilities

7.5 Key Milestones in 2026

• SK Hynix HBM4 Mass Production: First half of 2026
• Samsung HBM4 Mass Production: First half of 2026
• Micron HBM4 Mass Production: 2026 (specific timing TBD)
• NVIDIA Rubin Release: Second half of 2026
• AMD MI400 Series Release: 2026
• Global HBM Market Surpasses $58 Billion: 2026

Sources: Counterpoint Research (Apr 2026); Yole Group Next-Gen DRAM Report (Mar 2026); Samsung Press Releases; SemiAnalysis ISSCC 2026 Coverage

8. Conclusion

Since its standardization in 2013, HBM technology has undergone six generations of evolution, with per-stack bandwidth soaring from 128 GB/s to 2 TB/s, and capacity expanding from 4 GB to 64 GB. This evolution has fundamentally transformed the landscape of AI infrastructure—without the bandwidth and capacity support of HBM, the current large language models with parameters ranging from tens of billions to trillions could neither be trained nor efficiently inferred.

HBM4 represents the most transformative generation in this evolutionary process. Its 2048-bit interface is not merely a doubling of specifications, but a reconstruction of the entire memory subsystem architecture—affecting everything from channel configuration and interposer wiring to controller design. At the same time, HBM4 continues to use microbumps instead of hybrid bonding. This pragmatic decision reflects the severity of current yield and testing challenges and also reserves ample space for technology upgrades in the roadmap beyond HBM5.

From an industry landscape perspective, SK Hynix, by virtue of its early-mover advantage and deep partnership with NVIDIA, is currently in a position of significant leadership; Samsung is making an all-out effort to repair the damage caused by certification setbacks and has the potential to re-enter competition through HBM4; Micron, through technological breakthroughs, has achieved several-fold market share growth, becoming an undeniable third force.

The challenge of the Memory Wall will not disappear. Computing power grows 2–3× per generation, while HBM bandwidth grows 1.5–2× per generation—the gap will persist. But each generation of HBM evolution continuously pushes this ceiling higher: the roadmap from HBM5 to HBM8 is already in planning, and the introduction of new technologies such as hybrid bonding, processing-in-memory, and 3D DRAM will keep pushing this ceiling upward. In the investment race for AI infrastructure, HBM will continue to play the critical role of determining the upper limit of system performance.

References

1. Wikipedia. “High Bandwidth Memory.” https://en.wikipedia.org/wiki/High_Bandwidth_Memory
2. Introl Blog. “HBM evolution: from HBM3 to HBM4 and the AI memory war.” Feb 2026. https://introl.com/blog/hbm-evolution-hbm3-hbm3e-hbm4-memory-ai-gpu-2025
3. SemiEngineering. “HBM4 Sticks With Microbumps, Postponing Hybrid Bonding.” Jan 2026. https://semiengineering.com/hbm4-sticks-with-microbumps-postponing-hybrid-bonding/
4. PatSnap. “HBM technology landscape 2026: market and AI demand.” Mar 2026. https://www.patsnap.com/fr/resources/blog/articles/hbm-technology-landscape-2026-market-and-ai-demand/
5. PatSnap. “Thermal resistance in stacked DRAM: HBM4 solutions.” Apr 2026. https://www.patsnap.com/resources/blog/articles/thermal-resistance-in-stacked-dram-hbm4-solutions/
6. Counterpoint Research. “Hybrid Bonding Expands from Logic to Memory.” Apr 2026. https://counterpointresearch.com/en/insights/Hybrid-Bonding-Expands-from-Logic-to-Memory
7. Siemens Blog. “HBM3e and HBM4: IC design guide.” Apr 2026. https://blogs.sw.siemens.com/semiconductor-packaging/2026/04/24/hbm3e-hbm4-ic-design-guide/
8. SemiAnalysis. “Scaling the Memory Wall: The Rise and Roadmap of HBM.” Aug 2025. https://newsletter.semianalysis.com/p/scaling-the-memory-wall-the-rise-and-roadmap-of-hbm
9. SemiAnalysis. “ISSCC 2026: NVIDIA & Broadcom CPO, HBM4 & LPDDR6.” Apr 2026.
10. Blocks & Files. “DRAM and NAND: Micron and SK Hynix’s paths to production.” May 2025.
11. Aivon. “Comparing HBM, HBM2, HBM3 and HBM3e.” Feb 2026. https://www.aivon.com/blog/memory-storage-technology/comparing-hbm-hbm2-hbm3-and-hbm3e/
12. Micron. “HBM3E Product Brief.” https://www.micron.com/content/dam/micron/global/public/documents/products/product-flyer/hbm3e-product-brief.pdf
13. Yole Group. “Next-Gen DRAM 2026 - Focus on HBM and 3D DRAM.” Mar 2026.
14. Wevolver. “What is High Bandwidth Memory 3 (HBM3): Complete Engineering Guide 2025.”
15. Rambus. “High Bandwidth Memory (HBM): Everything You Need to Know.” 2024.
16. Astute Group. “SK hynix holds 62% of HBM, Micron overtakes Samsung, 2026 battle pivots to HBM4.” 2025.