Skeleton-based baseball pitch classification on broadcast videos

In baseball, accurate pitch type classification is essential for strategic decision-making by teams and analysts. Traditional methods rely heavily on ball trajectory tracking using radar and high-speed cameras, which are costly and accessible only in professional leagues. In contrast, this paper presents a skeleton-based approach to classify pitch types using pose estimation and spatial-temporal modeling. We extract key joint coordinates of the pitcher using OpenPose and model their body movements over time using a Spatial-Temporal Graph Convolutional Network (ST-GCN). Our method is evaluated on the publicly available MLB-YouTube dataset, achieving 68.2% accuracy in classifying six pitch types, and outperforming state-of-the-art methods that rely on full-frame data with 3D CNNs. By focusing exclusively on pitcher’s skeletal information through graph-based modeling, our approach improves classification performance, while showing robustness in binary tasks, reaching 85.7% accuracy for fastball detection and 80.8% distinguishing fast versus slow pitches. Our method’s performance underscores the effectiveness of relying on body mechanics for pitch classification, demonstrating that accurate results can be achieved without the need for costly ball trajectory data.