Seeing or understanding: Explaining the safety cognition failure with EEG and eye-tracking techniques

• Proposed a novel method for diagnosing internal cognitive failures using multimodal physiologic data. • Explored the neural mechanisms underlying hazard recognition and risk perception in construction. • Addressed the gap on perceiving risks and provide neural evidence of explanation of safety cognition. • Highlighted the significant role of professional experience in internalizing knowledge and improving safety cognition. • Provided a trained CNN-ResNet model for detecting risk perception failures in automatic monitoring of safety cognition. Cognition failure is a main cause of unsafe behavior, leading to accidents’ occurrence. Focusing on internal cognition processes is an effective way to manage external behaviors. However, the explanations of cognitive failure remain insufficient. This research explored the construction workers’ safety cognitive failure, by distinguishing the processes of hazard recognition and risk perception, as well as the role of professional experience in risk understanding, through experimental tests which combine the eye-tracking and EEG techniques for data collection and analysis. The eye movement data was used to identify cognitive failures occurred in seeing hazards; EEG data were combined with oral reports to detect cognitive failures in understanding risks; EEG provided higher-level cognitive information to reflect the characteristics of the brain activity at different stages of safety cognition. The obtained EEG results identified the β band as the key active frequency that associated with risk awareness, providing neurological evidence for cognitive process of understanding information. Moreover, it was found that experienced workers’ brain activities were more active when perceiving risks, which implied that professional experience plays an important role in improving the internalization of implicit safety knowledge. Furthermore, the time–frequency-power data from EEG was extracted and used to train a CNN-ResNet model for the detection of risk perception failure, laying a foundation for real-time automatic monitoring of safety cognition in construction sites in the future research. This study enriches the theory of cognitive process model and cognitive failure on risk perception and sheds light on targeted intervention of unsafe behavior.