A Multi-Sensor Late Fusion Framework for Driver Cognitive State Detection

The rapid advancement of technology has made modern vehicles highly technical, increasing driving speeds and in-car activities, yet road fatalities have risen due to unsafe driver behaviors. Ensuring driver alertness is essential for road safety, both in human-driven and autonomous vehicles. Cognitive load is a key measure of alertness, but real-world assessment is challenging, and wire sensor solutions are impractical. Recent developments in non-contact approaches using image processing offer promising alternatives. Driver cognizance detection, a focus area in the vision community, now benefits from multi-sensor devices capable of monitoring the driver’s cognitive state in day and night conditions. Our algorithm employs EO and NIR cameras to continuously capture images and fuse RGB and NIR data for accurate cognitive state detection, using temporal information to predict future states. These advancements, alongside declining hardware costs, enable non-intrusive, real-time driver monitoring, contributing to safer driving environments