報告_OCP2026_創意_AI運算記憶體連接擴展_20261015

PDF 原檔：報告_OCP2026_創意_AI運算記憶體連接擴展_20261015_original.pdf

講者公開簡報（OCP 官方 Recorded Content 頁提供下載）。文字以 PyMuPDF 逐頁抽取，### p<n> 為頁碼；圖表內容見原檔。

圖片清單（已驗證 2026-10-19）

原始內容

p1

Igor Elkanovich GUC CTO Scaling up AI Compute, Memory and Connectivity SERVER: OPEN CHIPLET ECONOMY

p2

DRAM Compute • Scaling up compute: 2.5D CoWoS® and 3D SoIC®-X • Memory: HBM5 and DRAM on Compute • Co-Packaged Optics (CPO) and COUPE • Vertical power delivery and Integrated Voltage Regulator (IVR) • System on Wafer Agenda Top Die Bottom Die

p3

• Compute • 3D integration (SoIC-X) is a key approach to increase compute density • High power density (5-10W/mm2) is a key challenge • Vertical power delivery to minimize loss • Voltage Regulators integration into PCB cavity → package substrate → CoWoS-L interposer • Cooling compute dies, memories (max 95C), CPO (temp. stability) and VRs is a big challenge Scaling up AI Compute, Memory and Connectivity • Memory • HBM5 doubles memory bandwidth vs. HBM4E • DRAM on Compute: architectural flexibility, high bandwidth, low latency and power • Connectivity: CPO • COUPE: high bandwidth, high radix and power efficient Optical Engines • COUPE supports both Scale Out and Scale Up

p4

• UCIe limitations • UCIe architecture is complicated – extra power and latency • In most of use cases standard compliance is not required Low Power & Latency 2.5D Die-to-Die Interface UCIe-S UCIe-A GLink-LL Speed 64Gbps 64Gbps 4-8Gbps Bump/Bond Pitch Standard package 110um CoWoS® 45um CoWoS® 35um SoIC®-X-CH 6um IP Depth 2mm 1.8mm 0.2-0.8mm Trace Distance 25-40mm 2mm 0.4-1.0mm Edge BW Efficiency 0.37 TBps/mm 2.6 TBps/mm 0.3-0.6 TBps/mm Area BW Efficiency 0.19 TBps/mm2 1.4 TBps/mm2 0.7 TBps/mm2 Power Efficiency 1.25 pJ/bit 0.75 pJ/bit 0.05-0.10 pJ/bit BER BER<1E-15 BER<1E-15 BER<1E-30 CRC+Retry CRC+Retry CRC+Retry Not required Latency with FIFOs 10ns + Retry 10ns + Retry 1-3ns • GLink-LL: low power & latency D2D • Moderate bandwidth • Power <0.1 pJ/bit • Latency: max 3ns including CDC FIFO • BER <1E-30, no CRC and Retry required

p5

• 3D interface is customized for every project • Bond map, DDR/SDR architecture, signal/clock ratio, redundancy, power domains, dimensions • GUC’s GLink-3D has been silicon-proven and adopted in customer’s AI ASICs • High bandwidth (80 Tbps/mm2), low power (0.03 pJ/b) and low latency (0.25ns) • Supports DVFS: voltage and frequency can change while keeping data transfer • Flexible IP delivery types: soft, semi-soft or hardmacro 3D Top-Bottom Dies Interface GLink-3D 1.0 Architecture: DDR Process: N6/N5 Power: 0.2 pJ/bit Bandwidth: 9 Tbps/mm2 Latency: 1-2ns Silicon Proven GLink-3D 2.0 Architecture: SDR, DDR Process: N2P/N2P Power: 0.03 pJ/bit Bandwidth: 50 Tbps/mm2 Latency: 0.125-0.5ns Silicon Proven GLink-3D 2.1 Architecture: SDR Process: N3P/N5 Power: 0.03 pJ/bit Bandwidth: 40 Tbps/mm2 Latency: 0.25ns Silicon Proven GLink-3D 3.0 Architecture: SDR, DDR Process: N2P/N2P Power: 0.03 pJ/bit Bandwidth: 80 Tbps/mm2 Latency: 0.125-0.5ns Execution * Full Duplex Bandwidth

p6

• Power delivery for high power density (7.5 W/mm² average TDP) solutions • Optimal distribution of bottom die’s TSVs to meet global TSV density • Optimizing TSVs channels between bottom die’s large SRAMs • In N2P TSV overhead was reduced by ~40% vs N3P increasing routing resources • Power and thermal analysis at early stage to avoid late stage iterations • Architecture and floorplan are adjusted for better power integrity and thermal • Mechanical strength: • Reducing warpage, package structure and chiplets floorplan optimization, materials selection 3D Key Challenges SoIC®-X Thermal Analysis Power Delivery Optimization Warpage Minimization Top Die Bottom Die

p7

• GUC’s HBM4E PHY/Controller at 16Gbps is ready • PHY supports face-up configuration for SoIC-X bottom die and top die’s logic over the PHY • Controller is optimized for high bandwidth utilization at random access • It supports advanced RAS functionality to prevent silent data corruption • HBM5 targets 24-32Gbps, will be ready for production in 2029 • HBM5 PHY area will be reduced by 40% by using smaller u-bump pitch • Thermal is a key challenge, HBM4E at 16Gbps is ~60W while HBM5 at 32Gbps will be ~90W • HBM5 D2D interface is the main heat source, solution: cooling silicon over D2D interface From HBM4E to HBM5 HBM4E HBM5-HBM5 Difference Speed per pin 16Gbps 24-32Gbps 1.5x-2x PHY size 14 mm2 8 mm2 0.6x PHY area efficiency 0.3 TB/mm2 0.75-1.0 TB/mm2 2.6x-3.5x Availability 2026 2029 Cooling Silicon over HBM’s D2D interface HBM5 vs. HBM4E Source: SK Hynix Newsroom May 2026

p8

• DRAM on Logic advantages: higher bandwidth, lower power and latency • HBM uses TCB 3D stacking with pitch ~20um (HBM4E) • Due to low TSV density HBM shares TSVs between banks • DRAM on Logic uses Hybrid Bonding (HB) 3D stacking with pitch 5-8um • Interface is 10-15x denser, separate wide buses per every bank are used • Maturity and cost • HBM is cost efficient for its long term manufacturing and improvements • HB stacking is not widely used yet but there is strong dynamics of HB adoption DRAM on Compute Logic DRAM on Compute HBM4E Total BW 32 TBps 24 TBps Memory density 4 layers: 110 MB/mm2 12 layers: 310 MB/mm2 Latency 30 ns 200 ns Power 0.5 pJ/bit 2.5 pJ/bit Max temperature 125C 95C * Including PHY and Controller HBM Stacking: TCB DRAM on Logic Stacking: Hybrid Bonding (HB) Full reticle size 4 layer DRAM **6xHBM4E at 16Gbps

p9

• Memory Controller is a key enabler for DRAM on Compute • GUC’s Memory Controller • Based on our HBM4E Controller, optimized for area efficiency and direct access to all DRAM banks • Supports multi-bank interleaving and access reordering for high bandwidth utilization • Interoperable with any DRAM vendor • DRAM yield management by banks redundancy and repair • Bandwidth utilization: ~100% at sequential access, 80%-70% at random access • Latency: <30ns from AXI read command to AXI read data Memory Controller for DRAM on Compute uB0 uB0 uB0 uB0 uBN uBN uBN uBN uB1 uB1 uB1 uB1 Layer0 Layer1 Layer2 Layer3 AXI Controller AXI to Banks mapping, Redundancy & Repair Bank0 DRAM Interface Bank1 Bank2 Bank3 BankN BankN-1 AXI 0 Compute Die DRAM Layers Memory Controller DRAM Interface Layer0 Layer1 Layer2 Layer3 AXI N ….. ….. ….. ….. ….. ….. …..

p10

• TSMC COUPE: high bandwidth, high radix, power efficient Optical Engines • TSMC COUPE’s tiny MRM/MRR and dense Fiber Attach (FAU) fit Slow-Wide • Optical Engines functionality: • Scale Out: IEEE-DR 200G/PAM4 per fiber, per direction • Scale Up, Slow-Wide OCI-gen2: 4xλ per direction 100G/NRZ • DWDM of 4xλ NRZ datastreams in a single bidirectional fiber • Area and power efficient NRZ modulation instead of PAM4 • Light, low latency FEC is used due to much lower NRZ BER TSMC COUPE for Co-Packaged Optics (CPO) GPU Tray GPU with CPO OCI Spec: DWDM channels for a Bidirectional Link

p11

• GUC’s COUPE manufacturing and testing flow uses robust and scalable ecosystem • Optical Engines are assembled on substrate aside CoWoS island • Four test insertion steps to ensure Optical Engines (OE) quality before XPU package assembly • Insertion 1: separate PIC and EIC wafer level tests • Insertion 2: PIC+EIC wafer level test after 3D stacking • Insertion 3: OE test after dies singulation • Insertion 4: Final test after entire xPU with OEs assembly with attached fiber and optical connector COUPE Manufacturing and Testing Flow TSMC (GUC) PIC Provider & GUC OSAT (GUC) TSMC, OSAT Customer ASIC or xPU Interposer WAT & inspection Electrical wafer sort CoW Memory (KGD) EIC PIC (SOI wafer) Electrical wafer sort Optical wafer sort OE integration E-O wafer sort Organic Substrate (MCP) Assembly on SBT Connector & FAU attach Final test Insertion#1 PIC & EIC(wafer level) Insertion#2 EIC+PIC(wafer level) Insertion#3 Singulated dies Insertion#4 Singulated OE integration & wafer sort EIC, PIC wafer sort Foundry CoWoS & MCP Assembly Final test Connector & FAU Optical Engine (OE) Assembly xPU with OE Assembly Si Carrier EIC PIC

p12

• For few kW ASICs vertical power delivery is a must to minimize loss • Voltage Regulators integration steps • PCB cavity → package substrate → CoWoS-L interposer • GUC’s IVR advantages • Low IR drop, Integrated inductor • Response time is 10x-100x faster than on-board VRM • DVFS per CPU/AI core • Current per ball and C4 bump (EM) is >2x lower • GUC demonstrated IVR using TSMC 16nm • Dual-phase IVR per CPU/AI core • Input voltage: 1.8V • Programmable output voltage: 0.5V-1.0V • Efficiency: 85% at high current, 75% at low current • Power density: 3W/mm2 Vertical Power Delivery Vertical Power Delivery IVR Integrated into CoWoS®-L Interposer VR at Land Side (Handheld Application)

p13

• Tens dies and HBMs in one wafer, for huge scale AI and networking applications • >40x reticle size of silicon area (>33,000mm2), up to 64 HBM4E, high speed IOs and connectors • UCIe interfaces replace SerDes: 10x less power, 20x lower latency • Key challenges: System on Wafer (SoW) 4x Quad GPU Servers Switch are replaced by one System on Wafer • Heat dissipation: 20kW power (TDP) • HBM max temperature: 95C • Power delivery: 30kW peak power • SoW enabling technologies: • UCIe 64Gbps Die-to-Die interface • 224Gbps routing using RDL layers • Power efficient implementation • Thermal/mechanical • Redundancy and repair • 3DIC and many more…

p14

• 3D significantly increases compute density, its key enablers: • High bandwidth, low power, low latency 3D interface • Power delivery for high power density and thermal optimization • Few generations of 3D products were validated in customer’s systems • Vertical power delivery supports high power density with minimal loss • IVR will be integrated in CoWoS-L interposer for fast response time and per-core DVFS • Memory • HBM4E PHY/Controller at 16Gbps is ready • HBM5 doubles memory bandwidth vs. HBM4E and increases PHY area efficiency • DRAM on Compute for flexible architecture, high bandwidth, low latency and low power • Connectivity: TSMC’s COUPE for CPO Optical Engine • COUPE enables high-bandwidth, high-radix and power-efficient system scaling • It is mature and supports both Scale Out and Scale Up requirements • System on Wafer: a game changer, 5x more silicon area and HBMs than the biggest CoWoS Summary

p15

（本頁無可抽取文字，內容在圖中）