Unified Collective Communication (UCC): An Unified Library for CPU, GPU, and DPU Collectives

Unified Collective Communication (UCC) is an API and library implementation of collective communication operations. The goal of UCC is to provide a unified API and library serving the collective communication needs of various workloads running on a wide variety of system architectures. Particularly, we aim to unify the collective communication interfaces and semantics and provide a common implementation framework for (i) parallel programming models including HPC (message passing and Partioned Global Address Space (PGAS)), Deep Learning (DL) and 1/0, (ii) collectives moving data in CPU main memory and device memory (GPU, and Data Processing Unit (DPU)), and (iii) collective operations using software point-to-point transports, and hardware transports. In this paper, we present an overview of UCC's design, interfaces, semantics, and an implementation. We demonstrate UCC's capabilities through evaluations with microbenchmarks representing diverse workloads and applications across various programming models including MPI, PGAS (OpenSHMEM), and PyTorch, on multiple hardware architectures (CPU, GPU and DPU). Our results show that with UCC, microbenchmarks perform up to 3.4X and 5.4X better on latency and bandwidth, respectively compared to the Tuned collective component in Open MPI. In addition, we show up to 12% performance improvement in applications without any modifications to the application itself and an up to 4X improvement when leveraging the flexibility UCC provides. These improvements highlight UCC's success in unifying collective communication needs. UCC has been adopted by multiple institutions, and it is currently integrated with Open MPI for Message Passing Interface (MPI) and OpenSHMEM programming models, and PyTorch for DL workloads.