An efficient temporal attention framework for real-time detection of unsafe driver behaviors

Driver distraction remains a leading cause of road accidents, underscoring the need for intelligent and resource efficient monitoring in modern transportation systems. A central challenge is the reliable recognition of subtle and overlapping distraction behaviors while maintaining the low latency required for real-time deployment. This work introduces the Attention Temporal Feature Extractor (ATFE), a lightweight framework that leverages attention-guided temporal-aggregation to capture both fine-grained cues and long-range dependencies from grayscale frame-to-sequence representations. The attention mechanism adaptively assigns greater weight to frames containing behaviorally discriminative cues such as brief hand object interactions or gaze diversions while suppressing neutral or redundant frames, thereby selectively emphasizing informative temporal segments. By selectively emphasizing informative temporal segments, ATFE enhances recognition accuracy while remaining computationally efficient and interpretable. Comprehensive evaluations on the public benchmark datasets NTHU-DDD and Drive&Act confirm ATFE’s effectiveness, achieving a mean F1-score of 94.9% ± 2.0% across three random seeds and 95.7% on the respective test sets. The model sustains real-time inference at 78 FPS with 12.8 ms latency, validating its suitability for embedded driver-monitoring systems. Beyond achieving strong accuracy, ATFE demonstrates robustness in distinguishing visually similar distractions and identifying rare unsafe behaviors, highlighting its promise as a practical and scalable solution for safety critical driver monitoring in advanced driver assistance and intelligent transportation systems. Figure: Overview of the proposed ATFE framework for real-time unsafe driver behavior detection, integrating frame-to-sequence construction, grayscale feature extraction, attention-guided temporal pooling, and majority voting to deliver robust video-level predictions. • A novel ATFE framework is proposed for real-time unsafe driver behavior detection. • Grayscale-optimized backbone reduces computational cost while preserving key cues. • Attention-guided temporal-aggregation improves recognition of subtle distractions. • Achieves 98.5% accuracy with 78 FPS and 12.8 ms latency on benchmark datasets. • Demonstrates robustness and scalability through ablation and comparative studies.