Contrastive Clustering for Cross-Modal Alignment and Distribution Bias in Visual Question Answering

In recent years, visual question answering (VQA) has become widely used in multimodal domains, the multimodal semantic gap and data distribution bias between modalities reduce model generalization, limiting improvements in comprehension and reasoning performance. To address these issues, we propose an algorithm termed Contrastive Clustering Algorithm(CCA), a coordinated framework that integrates contrastive learning with a clustering algorithm. CCA utilizes contrastive loss functions to construct cross-modal positive and negative sample pairs, enabling the effective mining and aligning of the semantic information between different modalities. At the same time, it combines with the clustering algorithm to divide the feature space at a fine-grained level, reducing intra-cluster differences and increasing inter-cluster separation for more discriminative feature representations. Extensive experiments on the VQA v2 dataset show CCA significantly enhances the cross-modal comprehension and reasoning ability in cross-modal scenarios, providing an effective approach and new strategies to mitigate semantic and distributional bias in VQA semantic and distributional bias in VQA.