Detection of Employee Fatigue Based on Image Processing Using Deep Learning Model

Fatigue in workplace is a common thing shared by all employees. Continuous exposure of fatigue could lead to negative productivity for companies. Current research on fatigue detection mostly focused to detect fatigues from a single 2D image data, meanwhile research on fatigue detection using each frames from video data is needed for better detection for fatigue process. The main advantage of using each timesteps of frames from video data which makes the model is able to predict long term dependency of fatigue person. To solve this problem, a deep learning model is proposed that could detect employee fatigue based on image processing using video data. Three models are used to train data using Convolutional Neural Network (CNN), which are Time Distributed CNN LSTM, 3D CNN, and 3D CNN LSTM. Out of those three models, the best model to detect fatigue person from video data is Time Distributed CNN Model with F-1 Score value of 0.77 and Accuracy Score of 0.76 for testing data. The model that gives best inference time is also Time Distributed CNN Model with the average value of 382 miliseconds to detect fatigues from 58 testing data with each of the data has duration of 3 seconds and 12 frames.