Feature Point-Driven Vision Transformer for Driving Behavior Analysis

针对Vision Tranformer (ViT)在局部特征捕捉和计算效率方面的局限性，文章提出了一种将目标检测技术和视觉神经网络分类模型融合到一起的方法。该方法针对驾驶行为分析特征点数量稀疏的特点，对ViT进行改进，提出了分阶段注意力计算策略，通过重构ViT编码层，将后五层的全局视觉特征序列替换为目标检测特征点序列，使其更适配基于特征点驱动的模型。并替换标准ViT的位置编码，引入方向盘相对距离–角度联合编码。此外，为了改善模型对面部捕捉的不足，在上述模型的基础上引入了多任务学习，加入了判断驾驶员面部是否直视前方这个子模型辅助判断，以更好地判断驾驶员的驾驶行为。这个模型称为特征点驱动的Vision Tranformer多任务学习模型(FViT-MTL)，在SFDDD数据集上的准确率为93.85%，比其他主流视觉神经网络分类模型提升了5.71%，比目前应用于驾驶行为分析先进的方法提升了1.28%，有效提升了分类模型的准确率，并确保驾驶行为的判断准确。To address the limitations of Vision Transformer (ViT) in local feature capture and computational efficiency, this study proposes an integrated approach that fuses object detection technology with visual neural network classification models. Specifically, we introduce a phased attention computation strategy tailored for sparse feature point scenarios in driving behavior analysis. By reconstructing ViT’s encoder layers, we replace the global visual feature sequences in the last five layers with target-detected feature point sequences, making them more suitable for feature-driven models. Additionally, we substitute standard ViT positional encoding with a combined relative position-angle encoding of the steering wheel to enhance spatial-temporal context understanding. Furthermore, to compensate for suboptimal facial feature capture, a multimodal subsystem is introduced through multitask learning to detect driver head orientation. The proposed Feature-Driven Vision Transformer Multitask Learning (FViT-MTL) achieves 93.85% classification accuracy on the SFDDD dataset, demonstrating a 5.71% improvement over conventional visual neural networks and a 1.28% gain compared to state-of-the-art methods. The results validate our approach’s effectiveness in achieving both computational efficiency and precise driving behavior analysis.