Multimodal prediction of situation awareness during automated driving: a gaze and EEG-based approach

Drivers' situation awareness (SA) is often assessed using indirect behavioural indicators such as takeover performance, despite known limitations. Many studies have also focused heavily on visual attention, primarily capturing the perception level of SA. To address this, we developed two physiological SA prediction models. Model 1 links physiological responses to <i>full-level SA</i> (perception, comprehension, and projection), while Model 2 targets <i>comprehension-level SA</i> to examine mechanisms beyond perception. A simulated driving study was conducted with 39 participants, during which eye-tracking and EEG data were collected. In Model 1, both modalities distinguished the degree of <i>full-level SA</i>. In Model 2, eye-tracking metrics differentiated successful from unsuccessful comprehension. In both models, the eye-tracking-based model outperformed the EEG-based model. However, combined model showed better performance, particularly in predicting <i>comprehension-level SA</i>. These findings demonstrate the benefit of integrating cognitive state monitoring beyond perception alone, especially for assessing higher-level SA in automated driving.