Facial Expression Recognition in Real-Time Surveillance Using CNN and Transfer Learning with ResNet-50

This paper describes the implementation of real-time FER system implemented using CNN with transfer learning from ResNet-50 for surveillance video analysis. The system is designed to efficiently identify seven basic facial expressions: segment by real-time video frames in the form of Happy, Sad, Angry, Surprised, Disgusted, Fearful and Neutral. Since there are lighting changes, occlusion and diverse orientations of the face in surveillance environment, the utilization of pretrained ResNet-50 allows the system to perform well and properly distinguish facial expressions. The proposed method uses the transfer learning to fine tune the ResNet-50 on a large FER dataset and obtain good classification accuracy, precision, recall, F1-Score as well. Evaluation data prove that the suggested model yields a 96.2% recognition rate as well as high precision (97.9%) and recall (96.9) ratios. In addition, inference time works as low as at 35 ms per frame to achieve 28.5 FPS for real-time application. Further, the proposed model provides better accuracy as compared to the regular models like the VGGNet and InceptionNet, along with less computation time. This work gives a strong real-time technique to recognize the facial expression which is prepared for use in security and surveillance systems.