AI-Based Employee Monitoring System

This paper proposes a local monitoring system for employees. It consists of collecting keyboard and mouse activity, monitoring running applications. Additionally, it involves the use of a web cam for detecting the presence of a face, saving the data to a backend, analysing data using a dashboard and machine learning classification. The information regarding keyboard and mouse events are collected using Python agent. We don't log the specific button pressed. Also, we log information about what is the current active window title, estimate how much time they spent on the approved application. And we check for the presence of at least one face. Every ~90 seconds we upload the observations of monitoring to FastAPI backend. Backend stores the data into an SQLite database using SQLAlchemy. React dashboard is used for giving the overall information about the employee data, score trend, face presence statistics, application details, filters, CSV exports. We try to send an email alert using smtp after ~60 seconds of not detecting the face. Data is analysed for a weighted score, and also Random forest classification is performed. The given dataset contains 1500 labelled (3classes) with 500 per class. an 80% -20% stratified split which resulted in a 91.33% accuracy for test set. This means that the model is consistent with the provided labels but it doesn't necessarily mean it can measure employee productivity. The system was built to enable contextual analysis.