Multimodal Multi-Scale Inception Network for Driver Attention Detection in Intelligent Vehicles

Researchers have made significant advancements in detecting driver attention using physiological signals, yet existing models are often constrained by weak feature saliency, narrow feature extraction scales, and reliance on single-modal data. This paper presents the Multimodal Multi-scale Inception Network (MM-IENET), a novel decision-making fusion model that integrates electroencephalography (EEG) and electrooculography (EOG) signals to address these challenges. The MM-IENET comprises an Inception Model (IE Model) and a decision-making fusion layer (Decision Layer). The crux of the IE Model is the Multi-scale Inception module, which extracts features across multiple scales, thereby enhancing feature diversity. To further augment feature saliency, six Multi-scale Inception units are stacked to form a Multi-scale Cooperative Module, increasing the model’s depth and capacity. Predictions from the EEG-based and EOG-based IE Models are subsequently fused in the Decision Layer using weighted ratios to yield the final classification. Extensive experiments on the SEED-VIG dataset validate the efficacy of our model. The classification accuracies achieved by the EEG-based IE Model, the EOG-based IE Model, and the proposed MM-IENET are 84.37%, 86.95%, and 90.16%, respectively, surpassing current state-of-the-art methods. These results underscore the model’s capability to efficiently extract data features and demonstrate the substantial performance gains attained through the fusion of EEG and EOG signals, fostering the advancement of Vehicle-to-Everything (V2X) technology in transportation systems.