ReCap: Event-Aware Image Captioning with Article Retrieval and Semantic Gaussian Normalization
Image captioning systems often produce generic descriptions that fail to capture event-level semantics which are crucial for applications like news reporting and digital archiving. We present ReCap, a novel pipeline for event-enriched image retrieval and captioning that incorporates broader contextual information from relevant articles to generate narrative-rich, factually grounded captions. Our approach addresses the limitations of standard vision-language models that typically focus on visible content while missing temporal, social, and historical contexts. ReCap comprises three integrated components: (1) a robust two-stage article retrieval system using DINOv2 embeddings with global feature similarity for initial candidate selection followed by patch-level mutual nearest neighbor similarity re-ranking; (2) a context extraction framework that synthesizes information from article summaries, generic captions, and original source metadata; and (3) a large language model-based caption generation system with Semantic Gaussian Normalization to enhance fluency and relevance. Evaluated on the OpenEvents V1 dataset as part of Track 1 in the EVENTA 2025 Grand Challenge, ReCap achieved a strong overall score of 0.54666, ranking 2nd on the private test set. These results highlight ReCap's effectiveness in bridging visual perception with real-world knowledge, offering a practical solution for context-aware image understanding in high-stakes domains. The code is available at https://github.com/Noridom1/EVENTA2025-Event-Enriched-Image-Captioning.
Authors
- Minh–Triet Tran (ORCID: https://orcid.org/0000-0003-3046-3041)
- Trung-Nghia Le (ORCID: https://orcid.org/0000-0002-7363-2610)
- Lam-Huy Nguyen
- Thanh-Hai Nguyen
- Thuan-Phat Nguyen
- Ginchev Dinh
Institutions
- Vietnam National University Ho Chi Minh City (VN)
- Ho Chi Minh City University of Science (VN)
Publication Details
- Published
- 2025-10-25
- DOI
- https://doi.org/10.1145/3746027.3762039
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00