Scalability of the ARM Nvidia Grace Superchip for Deep Learning Applications

The advance of Artificial Intelligence (AI) has heightened the demand for computational resources, presenting challenges in scalability and energy efficiency. The low-power ARM architecture is a promising option for AI workloads. This study assesses the scalability of the ARM Nvidia Grace Superchip in Deep Learning (DL) using a model that predicts referrals for severe diabetic retinopathy. We analyze performance in terms of speedup and energy consumption across 1 to 144 cores. Although speedup increases with more threads, the gains are less pronounced at higher core counts due to resource saturation. Energy consumption, however, significantly decreases, showing a 95% reduction when utilizing all cores. These results emphasize the Superchip’s potential for scalable and energy-efficient AI, especially in demanding applications. Future research will focus on memory usage, latency, and distributed learning.