Design and Analysis of Ultra High Bandwidth (UHB) Interconnection-based GPU-Ring for the AI Superchip Module

In this paper, we propose an ultra-high bandwidth (UHB) graphic processing unit (GPU)-ring structure for next-generation superchip modules. The interconnection via switches in multi-GPU system involves high latency and power consumption, which undermines the performance of artificial intelligence (AI). To address this issue, UHB GPU-ring architecture connects 4 GPU-high bandwidth memory (HBM) modules with short interconnection and high data rate on the same computing node. Proposed UHB GPU-ring architecture integrates the GPU-HBM modules directly, which enables to function 4 GPU-HBM modules as 1 module, decreasing the non-uniform memory access (NUMA) effect. To ensure the signal integrity of the proposed UHB GPU-ring, we designed the hardware system with the continuous time linear equalizer (CTLE) and decision feedback equalizer (DFE) at the receiver referring to Peripheral Component Interconnect Express (PCIe) 5.0 system. As a result, we verified 32TB/s bandwidth and 0.6 pJ/bit between GPU-HBM modules, which enhances bandwidth by 17.7 times and energy consumption by 2.5 times. The result showed that the UHB GPU-ring can be a promising solution for the next generation superchip module.