Improving overall GPU sharing and usage efficiency with Kubernetes

GPUs and accelerators are enabling High Energy Physics (HEP) to keep pace with the growing data volume and computational complexity. The challenge remains to improve overall efficiency and sharing opportunities of what are currently expensive and scarce resources. In this paper, we describe the common patterns of GPU usage in HEP, including spiky requirements with low overall usage for interactive access, as well as more predictable but potentially bursty workloads. We then explore the multiple mechanisms to share and partition GPUs, covering time-slicing, and physical partitioning (MIG) for NVIDIA devices. We conclude with the results of an extensive set of benchmarks for representative HEP use cases. We highlight the limitations of each option and the use cases where they fit best. Finally, we cover the deployment aspects and the different options available targeting a centralized GPU pool that can significantly push the overall GPU usage efficiency.