Heterogenous Integration in High Volume: Now and Looking Forward

AMD is an industry leader in the use of advanced packaging to enhance its products. Over the past decade, this has included volume production of 2.5D technologies including silicon interposer, organic RDL interposer (FO-RDL), and embedded fan-out bridge (EFB); hybrid bond (SOIC) 3D logic-on-SRAM stacking, and cost-effective chiplet re-use with multi-chip modules. The recently released AMD Instinct MI300 AI chip sets a new standard for what can be achieved from heterogenous integration with 3.5D packaging 3D hybrid bond logic die stacks are connected together and to eight total eight- or twelve-high HBM DRAM stacks using a 2.5D silicon interposer, resulting in a module with massive compute capacity and the near memory capacity and bandwidth needed to support it.This presentation will begin with a brief review of the history of AMDs advanced packaging engagements and the technical incentives for investing in specific technologies. It will then transition into a deeper look at the AI-specific trends that resulted in 3.5D packaging technology. These include compute performance doubling faster than every two years, memory bandwidth doubling every two years, networking bandwidth doubling at only a slightly slower pace, and total power consumption continuing to rise. How heterogenous integration benefits each of these areas will be discussed, along with some of the challenges it introduces. The presentation will conclude with a look at how these trends scale into the future and where AMD is investing its efforts in next-generation technologies. Continuous improvement in 2.5D and 3D capabilities through pitch scaling, increased stacking height, and incorporation of additional features such as high density capacitors remains critical to meet next-generation targets. The evolution of silicon process will also factor into the way next-generation modules are designed, as AI workload power densities strain against the limits of what even advanced liquid cooled thermal solutions can support. The extremely large size of AI modules, their power consumption and growing off-package IO bandwidth requirements necessitate innovations in the rest of the packaging ecosystem as well, as SMT warpage, power delivery, and high-speed IO all have fundamental issues meeting current scaling trends. These challenges ensure there are many important problems for the advanced packaging community to solve in the decade to come.