Change Entity-guided Heterogeneous Representation Disentangling for Change Captioning

Change captioning aims to describe differences between a pair of images using natural language.However, learning effective difference representations is highly challenging due to distractors such as illumination and viewpoint changes.To address this, we propose a change-entity-guided disentanglement network that explicitly learns difference representations while mitigating the impact of distractors.Specifically, we first design a change entity retrieval module to identify key objects involved in the change from a textual perspective.Then, we introduce a difference representation enhancement module that strengthens the learned features, disentangling genuine differences from background variations.To further refine the generation process, we incorporate a gated Transformer decoder, which dynamically integrates both visual difference and textual change-entity information.Extensive experiments on CLEVR-Change, CLEVR-DC and Spot-the-Diff datasets demonstrate that our method outperforms existing approaches, achieving state-of-the-art performance.