Efficient Real-Time Fine-Grained Action Recognition over a Progressive and Hierarchical Classification Framework

Real-Time fine-grained action recognition (AR) presents significant challenges in resource-constrained environments with strict accuracy requirements. This paper proposes an efficient real-time AR system that utilizes a progressive hierarchical classification framework to achieve high accuracy while minimizing computational demands. The system utilizes the YOLO model for initial single-frame classification, enabling precise identification of alarming actions with a high recall rate to facilitate timely alerts. Subsequently, a second-tier recognizer that relies on spatiotemporal features is applied for fine-grained AR of identified alarming actions. To enhance recognition accuracy, we introduce a hierarchical classification model where actions are grouped based on semantic and kinematic similarity, followed by further classification within each group. Additionally, we implement a multi-threaded scheduling pipeline that ensures prompt alarms with reasonable loading time for precise AR. Experimental results demonstrate that our system effectively balances computational efficiency with recognition accuracy, making it suitable for real-time deployment in resource-constrained settings.