ADTC: Adaptive Dual-Stage Tree Construction for Point-Supervised Video Moment Retrieval

Video Moment Retrieval (VMR) is a key cross-modal task with broad theoretical and practical applications. While fully supervised methods deliver strong performance, they are constrained by the high cost of temporal boundary annotations. Weakly supervised methods mitigate this issue but suffer from limited accuracy due to coarse supervision. Recently, point-supervised approaches that leverage single-frame annotations as a cost-effective alternative have emerged as a promising paradigm. However, these methods often fail to leverage annotated frames for cross-modal semantic alignment. Additionally, they overlook global video structures and hierarchical segment relationships, leading to suboptimal retrieval accuracy under sparse supervision. To address these challenges, we propose the Adaptive Dual-Stage Tree Construction (ADTC) model, a novel framework designed specifically for point-supervised VMR. First, the model introduces a dual-stage hypothesis tree architecture that seamlessly integrates local and global trees, enabling the effective modeling of semantic relationships across multiple temporal scales. Second, it incorporates frame clustering and scene segmentation to extract the structural characteristics of video content, providing a foundation for comprehensive node relevance evaluation and an adaptive merging control strategy to optimize tree construction. Third, a hierarchical adaptive tree pruning strategy is implemented, combined with a novel proposal selection mechanism for distinguishing between positive and negative samples. These components are jointly optimized through a multilevel loss function, enabling enhanced semantic alignment and retrieval performance. The experimental results demonstrate that ADTC achieves state-of-the-art performance on the Charades-STA and ActivityNet Captions datasets under the point-supervised setting. On Charades-STA, it attains R@1 scores of 50.28% at IoU = 0.5 and 34.79% at IoU = 0.7, outperforming existing point-supervised methods. On ActivityNet Captions, it achieves R@1 scores of 65.02% at IoU = 0.3 and 46.13% at IoU = 0.5, establishing new benchmarks. Notably, it approaches fully supervised performance while significantly reducing annotation costs. Ablation studies further confirm the effectiveness of each model component.