MVRCap: Towards Comprehensive Video Captioning via Hierarchical Multi-Dimensional Verifiable Rewards
Despite significant progress, current models struggle to generate comprehensive video captions, constrained primarily by coarse-grained supervision, which often fails to capture spatial-temporal dynamics and the underlying logical intent. To address this issue, we introduce MVRCap, a novel framework that reformulates video captioning as a reinforcement learning task steered by structured verification. Central to our approach is a Fine-Grained Verifiable Reward mechanism, serving as dense supervision to enforce rigorous visual-semantic alignment. Specifically, we construct this reward system anchored by three complementary pillars: a Hierarchical Concept Coverage Reward utilizing our newly constructed MVRCap-QA dataset to ensure multi-dimensional completeness and semantic depth, a Factuality Verification Reward to enhance factual consistency, and a Conciseness Constraint to maximize semantic density. Furthermore, we curate the high-quality MVRCap-SFT dataset, providing structured annotations across six key dimensions, to serve as a robust foundational initialization for the RL policy in this training pipeline. Extensive experiments demonstrate that MVRCap achieves state-of-the-art performance on authoritative benchmarks including VDC, CAPability, and VC4VG.
Authors
- Xianing Chen
- Bo Zheng (ORCID: https://orcid.org/0000-0001-6630-7396)
- Rongrong Ji (ORCID: https://orcid.org/0000-0001-9163-2932)
- Congzhi Zhang
- Xiaoshuai Sun
- Zhibin Wang
- Miao Chen
- Jiayi Ji
- Yiwei Ma
- YuCheng
- Jun Song
- Weihuang Lin
Publication Details
- Journal
- arXiv (Cornell University)
- Published
- 2026-10-27
- DOI
- https://doi.org/10.48550/arxiv.2610.36838
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- preprint