Co-designing 2.5D Silicon Photonic Accelerators for Distributed Transformer at the Edge

The efficient execution of attention-based transformers and large language models on traditional CPUs and GPUs presents significant challenges related to performance and energy efficiency. While innovative solutions like ASICs, FPGAs, and ReRAMs have been explored, the field of silicon photonics has emerged as a promising avenue for developing energy-efficient accelerators for deep AI models. Notably, existing endeavors in silicon photonics have predominantly concentrated on inference for deep AI algorithms, leaving a limited number of initiatives focused on creating comprehensive deep learning accelerators capable of real-time training for transformer-like algorithms. This paper utilizes the superior merits of silicon photonics to realize a full-fledged transformer accelerator equipped for both inference and training. Introducing PHOTRAN, an AI analog photonics accelerator, we harness silicon microdisk-based convolution, photonic phase-change memory-based cache, and dense-wavelength-division-multiplexing to achieve energy-efficient and ultrafast transformer acceleration. Through evaluations using a commercial CAD framework on benchmark models, including Vision Transformers and Large Language models, our results showcase the superior performance of PHOTRAN. This work underscores the significant potential of photonic computing for on-chip training of large deep AI models.