Deep LearningDriven RealTime Worker Monitoring System for Fatigue, Stress, and Safety Risk Analysis

Extended working hours and repetitive industrial tasks significantly contribute to worker fatigue and stress, increasing the likelihood of workplace accidents and reducing productivity. This paper presents an advanced Real-Time Worker Fatigue and Stress Detection System that leverages deep learning, computer vision, and multimodal analytics for proactive risk monitoring. The proposed system integrates a FastAPI-based backend with a React frontend to perform real-time analysis of video and audio inputs. Unlike traditional systems, it employs YOLOv8 for high-speed multi-face detection, with MediaPipe as a fallback mechanism to ensure robustness. Facial regions are enhanced using CLAHE and Retinex preprocessing to handle illumination variability. Emotion classification is performed using a CNN trained on the FER-2013 dataset and refined using a Temporal Transformer model to capture sequential dependencies. The system further incorporates Deep SORT for multi-person tracking and DeepFace for identity recognition, enabling personalized monitoring. A multimodal pipeline integrates voice-based stress detection using acoustic features. A dynamic risk scoring engine computes worker risk levels using weighted contributions from fatigue, stress, and session duration, with worker-specific calibration. MongoDB is used for real-time and historical analytics. The system processes data in memory to ensure privacy. Overall, the solution provides a scalable, intelligent, and non-intrusive approach to workplace safety.