CMS2-Fusion: Cross-Modal State Space Coupling for Infrared and Visible Image Fusion
Abstract Infrared and visible image fusion aims to integrate thermal radiation information and scene texture details into a single comprehensive representation. Existing deep learning methods typically encode the two modalities through separate network branches and perform fusion only at the feature level, overlooking the potential benefits of cross-modal interaction during encoding. Inspired by state space models, this paper proposes CMS 2 -Fusion, which introduces a cross-modal state space coupling mechanism that allows the hidden states of both modalities to dynamically interact within the Mamba block through a learnable coupling coefficient, modeling thermal-texture dependencies at the state-transition level. On the MSRS dataset, CMS 2 -Fusion achieves a Q abf of 0.7704 and an SSIM of 0.6562, surpassing classical CNN methods and a decoupled Mamba baseline, while reducing parameters by 13% (8.4M vs. 9.6M) through shared SSM parameters. This work is the first to apply state-space cross-modal interaction to infrared-visible fusion, realizing a paradigm shift from “encode-then-fuse” to “encoding-as-coupling.”
Authors
- Hanfei Zhang
- Jiajia Liu
Institutions
- Huaiyin Normal University (CN)
Publication Details
- Journal
- Research Square
- Published
- 2026-11-12
- DOI
- https://doi.org/10.21203/rs.3.rs-10966967/v1
- Primary Topic
- Advanced Image Fusion Techniques
- Type
- preprint
Funders
- Huai'an Science and Technology Bureau