Smart Surveillance in Retail Stores: Detection, Classification, and Documentation of Suspicious Behavior Utilizing YOLOv11 and XGBoost

Retail theft persists as a prevalent challenge, often compounded by the limitations of traditional surveillance systems that heavily depend on manual human oversight. This study propose an automated surveillance system design to mitigate this issue by detecting and documenting suspicious customer behavior. Utilizing YOLOv11 Pose Estimation for human keypoint detection and XGBoost classification for behavior analysis, the system catagorizes movements as “Normal” and “Suspicious”. Crucially, it automatically captures screenshots and timestamps upon incident detection, providing, structured evidence. Performance evaluation revealed a robust model, with an accuracy of 94.35 %, precision of 95.59 %, recall of 94.20 %, and F1-Score of 94.89 %. The successfully validated model was then deployed in a practical surveillance environment, where it demonstrated its capacity for automated detection and effiecient documentation from video footage, producing valuable image and structured CSV log outputs. These features are instrumental for strategic analysis, including optimizing camera placement and enchancing workforce situational awareness in retail security. Future research should prioritize reducing false positives by integrating richer contextual visual information, such as shopping bags, product shelves, or nearby personnel, to imporve classification reliability in diverse real-world scenarios.