STAA: Spatio-Temporal Attention Attribution for Real-Time Interpreting Transformer-Based AI Video Models

Transformer-based video models have performed SOTA in various action recognition and video understanding tasks. The limitation is the lack of explainability of these complex models. Current Explainable AI (XAI) methods focus on feature importance analysis of only one dimension, either spatial or temporal features. When applied to transformer-based video models, the challenges are two-fold. They fail to capture the integrated spatio-temporal nature of video data and incur prohibitive computational costs for real-time applications. This paper presents STAA (Spatio-Temporal Attention Attribution), an XAI method for interpreting video transformer models for action recognition tasks. STAA simultaneously extracts both spatial importance within frames and temporal relevance across the video sequence from attention values in transformers. This unified approach solves the challenge by capturing how transformer models integrate information across both dimensions for decision-making, providing comprehensive explanations that reflect the model’s actual reasoning process. We enhance STAA’s raw output through post-processing. The experiments on the Kinetics-400 dataset demonstrate superior faithfulness (0.844 ± 0.116) and monotonicity (0.850 ± 0.030). In terms of computational overhead, our STAA method is applicable for real-time video XAI analysis applications. We implement a cloud-based architecture that enables video explanations with 150 ms latency.