Enhancing Multimodal Analogical Reasoning Through Triplet Interaction

Analogical reasoning is fundamental to human cognition and plays a crucial role across various fields. However, previous studies have primarily focused on single-modal analogical reasoning, often ignoring the benefits of incorporating structural knowledge. Research in cognitive psychology has shown that information from multimodal sources brings more powerful cognitive transfer than single-modality sources. Motivated by the cognitive analogical theory, we address the challenge of multimodal analogical reasoning over multimodal knowledge graphs. Our approach introduces a novel triplet interaction mechanism, where both internal and external interactions between entities and relations are leveraged to enhance the accuracy of multimodal analogical reasoning. Experimental results show that our model outperforms Transformer-based multimodal pre-trained baselines. This improvement underscores the essential role of triplet interactions in integrating multimodal data, leading to more accurate analogical reasoning processes.