Research on Multi-Attention-Based Multimodal Fusion Model

Most current fusion model studies focus on the impact of different modalities while ignoring the effect of inter-modal interaction information. This paper proposes a multi-attention-based multimodal fusion model. This model fully explores and integrates multimodal information and inter-modal interaction information through three attention modules. It extracts effective interaction information from the three attention modules using attention mechanisms, filtering out redundant and noisy data to improve the utilization of inter-modal interaction information and thus enhancing the accuracy of fusion model. Experiments conducted on two public datasets CMU-MOSI and CMU-MOSEI demonstrate this model achieves better fusion performance.