Human Activity Recognition Using Deep Sort

Abstract—Human Activity Recognition (HAR) is a key area in computer vision with applications in surveillance, healthcare, and smart environments. This paper presents a real-time HAR framework that integrates lightweight object detection, multi-object tracking, and rule-based activity classification. The system uses YOLOv8n for detecting humans in video frames and YOLOv8n-pose for extracting pose keypoints. Deep SORT is employed for multi-person tracking, ensuring consistent identity assignment across frames. A sliding window of pose keypoints is maintained for each tracked individual, and a set of handcrafted rules are used to classify activities such as standing, walking, and hand waving. The use of a rule- based classifier eliminates the need for complex temporal models, allowing efficient inference on standard hardware. The system generates annotated video output with bounding boxes, IDs, and activity labels, offering both interpretability and high responsiveness. Results from real-world video sequences demonstrate the system's effectiveness in recognizing actions and preserving identity continuity in dynamic environments. Index Terms - YOLOv8, Pose Estimation, Deep SORT, Multi-Object Tracking, Real-Time Video Processing, Rule-Based Classification, Identity Tracking, Video Analytics, Computer Vision, Behaviour Recognition, Smart Surveillance.