Multimodal cultural heritage image recognition based on quantum and classical multimodal fusion network

Existing studies mainly depend on visual features to construct deep learning-based models to recognize Cultural Heritage (CH) images while facing the challenges of insufficient feature expressiveness, increasing algorithmic complexity, higher requirement of computational resources, etc. To address these limitations, we introduce quantum computing technology and propose Quantum-Classical Multimodal Fusion Model (QCMFM) for CH image recognition aggerating visual features of CH images and textual features of CH image captions. The proposed model QCMFM is capable to fully synthesize the advantages of quantum and classical neural networks and effectively capture multimodal interactions and correlations. Experimental results on two constructed multimodal CH image recognition datasets demonstrate that QCMFM is superior compared with multiple strong baseline models. This study showcases that the quantum computing is capable to bring new possibilities and opportunities in the landscape of methodologies in CH, having great potential to be “a new road to Rome” in resolving CH tasks.