PassAI: An Explainable Machine Learning Framework for Predicting Soccer Pass Outcomes Using Multimodal Match Data

As sports data, increasingly shaped by recent advances in in-game data acquisition technologies, become more complex and high-dimensional, analyzing such multimodal datasets presents challenges in both predictive performance and interpretability. Therefore, in this study, we introduce PassAI, a novel machine learning framework for classifying soccer passes success or failure using spatiotemporal tracking images and player-specific statistical profiles. PassAI addresses these challenges with an original dual-stream architecture that processes image-based and tabular features separately and then combines them for final prediction. Additionally, a two-stage explanation module was implemented to provide interpretable insights from both inter- and intra-modality perspectives. This module allows end users (e.g., coaches and analysts) understand whether pass outcomes depend more on contextual dynamics or individual player traits. In experiments involving 6,349 passes from professional Japanese soccer league games, PassAI outperformed state-of-the-art models—including convolutional neural network-based and graph neural network-based approaches, as well as existing pass prediction models—by more than 5% in accuracy. It also provided interpretable feedback through visual saliency maps and feature sensitivity analysis. Furthermore, RemOve And Retrain was also performed to verify the faithfulness of the generated explanation, and the visual saliency maps were highly faithful. Although there remains room for improvement from a practical standpoint, the results of this study indicate the potential of explainable multimodal artificial intelligence systems in real-world, high-stakes decision-making environments such as professional sports.