Employee monitoring system with computer vision: Face and activity recognition

Vision-based surveillance system is used by major counties to monitor public spaces, namely face recognition to identify people and activity recognition to detect unwanted behaviour. The paper proposes a flexible and lightweight monitoring pipeline that runs both face and activity recognition at a time and is aimed for convenient deployment on closed spaces like schools or offices. The pipeline leverages popular vision models such as YOLOv8 for object detection, InsightFace for face recognition, and a vision- language model, CLIP, for zero-shot activity recognition that won’t need any model training. Testing the accuracy is done using two office footages, each with different complexities of camera angle and crowd distribution, one being far more complex than the other. The result is that YOLOv8 peaked at 95,73% accuracy, InsightFace at 96,56% accuracy, and CLIP at 53,25%. The testing provided an insight that the pipeline’s accuracy deteriorates at more complex footages with varying distances and angles of the object, and further research are needed on improving the pipeline’s optimization, known faces processing, and camera positioning.