EXPERT: An Explainable Image Captioning Evaluation Metric with Structured Explanations
Recent advances in large language models and vision-language models have led to growing interest in explainable evaluation metrics for image captioning.However, these metrics generate explanations without standardized criteria, and the overall quality of the generated explanations remains unverified.In this paper, we propose EXPERT, a reference-free evaluation metric that provides structured explanations based on three fundamental criteria: fluency, relevance, and descriptiveness.By constructing large-scale datasets of highquality structured explanations, we develop a two-stage evaluation template to effectively supervise a vision-language model for both scoring and explanation generation.EXPERT achieves state-of-the-art results on benchmark datasets while providing significantly higherquality explanations than existing metrics, as validated through comprehensive human evaluation.
Authors
- Sungzoon Cho (ORCID: https://orcid.org/0000-0002-1695-1973)
- Jongheon Jeong (ORCID: https://orcid.org/0000-0002-4058-5774)
- Sangyeop Kim (ORCID: https://orcid.org/0000-0002-7584-1061)
- Hyunjong Kim (ORCID: https://orcid.org/0000-0001-6254-663X)
- Yeongjae Cho
Publication Details
- Published
- 2025-01-01
- DOI
- https://doi.org/10.18653/v1/2025.findings-acl.1367
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- National Research Foundation
- National Research Foundation of Korea
- Ministry of Science and ICT, South Korea