Generalizing Driver Behavior Recognition in Complex Driving Scenarios via Hierarchical Attention Network with Hybrid-Split Strategy

Distracted driving represents the primary causal factor in traffic fatalities, necessitating the deployment of driver behavior recognition systems for real-time behavioral monitoring as a critical component of intelligent transportation systems. However, existing researches exhibit critical limitations in generalization: insufficient consideration of data set attributes such as scenario diversity, and driver group size during construction; reliance on conventional splitting strategies that either produce inflated results by random-splitting or underfitting via driver-splitting; and difficulty in identifying behaviorally salient regions due to their sparse spatial distribution within driving scenes. To address these limitations, first, we construct a large-scale multimodal data set named SAA13, which encompasses diverse scenarios and driver groups, providing a more realistic representation of real-world driving conditions. Additionally, we introduce a hybrid random-driver-split strategy that maintains interdriver variability in training subsets while ensuring consistent intradriver representation during testing, thereby enhancing generalizability through distribution-preserving augmentation. Furthermore, we propose a hierarchical attention network (HAN) with dual processing pipelines, including a coarse-grained spatial distillation module for context prioritization and a fine-grained recognition module for multiscale feature aggregation. Extensive experiments are carried out and demonstrate the necessity of our hybrid-split strategy in training and generalizing models in complex driving scenarios. Importantly, our HAN achieves 94.33% recognition accuracy with a 33.2 FPS rate on the trained SAA13 benchmark, 99.24% recognition accuracy under extremely poor lighting conditions, and 81.09% generalization accuracy on the unseen SAM-DD data set in the cross-data set test, suppressing other state-of-the-art (SOTA) models. These findings validate the exceptional effectiveness and robustness of our HAN in handling and generalizing real-world driving complexities.