Deep Learning and Machine Learning Based Automated Person Identification with Intruder Detection for CCTV Surveillance

With the evolution of technology, human beings increasingly depend on its diverse applications to enhance daily activities, improve efficiency, and ensure safety. In the realm of security, technological advancements have led to the development of sophisticated surveillance systems to automate tasks traditionally performed by humans. This research focuses on the development of an automated unknown person identification system for Closed-Circuit Television (CCTV) that leverages deep learning and machine learning algorithms to enhance security and surveillance capabilities. By utilizing the You Only Look Once version 8 (YOLOv8) model for person detection, this system employs a suite of models, including Deep Neural Networks (DNN), Multi-task Cascaded Convolutional Networks (MTCNN), the Viola-Jones Framework, and Histogram of Oriented Gradients (HOG) for face detection. We utilize deep learning and machine learning algorithms to improve our system’s performance for face recognition. FaceNet, VGGFace2, DeepFace, and the Face Recognition library are used to convert the face images into higher-dimensional embeddings. Then, to classify these embeddings, several machine learning algorithms such as K-Nearest Neighbors (KNN), Support Vector Machines (SVM), Decision Trees, Random Forest, and Extreme Gradient Boosting (XGBoost) are used. In face detection, DNN outperforms other models in terms of high detection rate and accuracy. For face recognition, KNN and SVM with FaceNet embeddings achieve an accuracy of 96.93% and 93.86% respectively, and for VGGFace2, both KNN and SVM achieve an accuracy of around 90%. However, in terms of a lower misclassification rate for unknown classes, the best model is SVM with FaceNet embeddings.