Hybrid Multimodal AI Framework for Real-Time Employee Activity Recognition and Workplace Motion Analytics

In modern workplaces, finding a balance between employee productivity and privacy is a significant challenge. The Hybrid Multimodal AI Framework for Real-Time Employee Activity Recognition and Workplace Motion Analytics (HMAI-ERA) is a novel system that employs computer vision, motion tracking, and behavioral analysis to monitor employee activity in real time. The framework incorporates face identification (FaceNet & MTCNN), screen usage monitoring, mouse and keyboard activity tracking, and movement tracking to tag employees as "working" or "not working." Logs of activities with timestamps are saved in MongoDB for advanced analytics and visualization with an admin dashboard implemented using Flask. The framework also detects recurring mouse movement and idle screen usage to mark for potential anomalies. By combining several modalities, HMAI-ERA offers a non-intrusive yet effective way of monitoring workplace productivity. Experimental results indicate that the system can successfully monitor employee engagement with low false positives. The proposed framework offers a scalable, real-time, and privacy-aware solution for modern businesses seeking data-driven workplace insights.