SaccadeAction: A Fast and Accurate Action Tubelet Detector Based on Center and Corner Keypoints

Existing efficient algorithms for action tubelet detection focus solely on the central regions of action instances and then predict the action locations. However, one issue is that these central key points can be far from the edges of the action instance, making it difficult to precisely define action boundaries in some scenarios. To address this, this paper proposes a fast and accurate action tubelet detector named SaccadeAction, which effectively focuses on both the center key point and corner points of an action instance to achieve a coarse-to-fine action tubelet prediction. SaccadeAction consists of the following four core head branches:(1) Center branch, responsible for predicting the central position of the action and its category;(2) Motion branch, which estimates the movement trajectory of the center in the temporal dimension by associating adjacent frames;(3) Box branch, which first predicts the approximate positions of the corner points within the bounding box, then combines the feature information from the center point and corner points to refine the instance boundary box;(4) Corner branches, which are only used during the training phase to extract more informative corner features, thereby improving performance without introducing additional computational costs during inference. These four branches work together to output tubelet detection results, which can be further associated with video-level action tubes through matching strategies. Experimental results on the JHMDB dataset show that SaccadeAction not only achieves higher video-mAP performance but also maintains a similar inference speed compared to methods that rely solely on central key points for action detection.