AthenaTriton: A tool for running machine learning inference as a service in Athena

Machine learning (ML)-based algorithms play increasingly important roles in almost all aspects of the data analyses in ATLAS, including detector simulations, event reconstructions, and data analyses. These diverse ML models are being deployed in the ATLAS software framework, Athena. To harmonize the ML inference in both the Athena environment and the ROOT environment, a dual-use ML interface is defined and implemented with two distinct backends: the Open Neural Network Exchange (ONNX) Runtime and Inference as a Service. While ONNX Runtime serves as the primary inference backend in Athena, scalable inference strategies are required to address the growing demands of processing simulation and collision data, including maximizing event throughput and utilizing coprocessors like graphics processing units (GPUs). To meet this challenge, we introduce AthenaTriton, a solution that integrates the NVIDIA Triton Inference Server with Athena. In AthenaTriton, Athena operates as a Triton client that sends requests to a remote or local server that performs the model inference. This scalable approach can be used in both online and offline computing workflows.