Real-World Spectral Compressive Imaging by Learning a Physics-Informed Forward Model
Coded aperture snapshot spectral imaging (CASSI) enables snapshot acquisition of hyperspectral image (HSI) by compressing a three-dimensional spectral data into a single two-dimensional measurement. While recent learning-based reconstruction methods achieve impressive performance on simulated datasets, their performance often degrades significantly on real CASSI systems, revealing a simulation-to-reality gap. This gap arises from a mismatch between the complex real imaging process and the reconstruction mappings learned from idealized simulated data. To bridge this simulation-to-reality gap, we propose a learnable physics-informed forward model (LPFM) that incorporates the physical structure of CASSI into a learnable proxy network to model the complex real imaging process. To train the LPFM, we construct the first real-world spectral compressive imaging (rSCI) dataset. The rSCI dataset contains 219 pairs of static HSI-CASSI images across diverse indoor and outdoor scenes, as well as additional dynamic CASSI videos. Leveraging rSCI and LPFM, we establish the first real-world benchmark for CASSI reconstruction. Extensive experiments demonstrate that incorporating data generated by LPFM effectively reduces the simulation-to-reality gap and consistently improves reconstruction performance across various state-of-the-art methods.
Authors
- Xiangtian Ma
- Lingqin Kong (ORCID: https://orcid.org/0000-0002-5815-7698)
- 宋维涛 (ORCID: https://orcid.org/0000-0001-9350-7107)
- Xin Wang (ORCID: https://orcid.org/0000-0001-9546-6038)
- Yuchen Wang (ORCID: https://orcid.org/0000-0002-1077-5294)
- Hua Huang
- Lizhi Wang
Institutions
- Beijing Institute of Technology (CN)
- Beijing Normal University (CN)
Publication Details
- Journal
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2026-11-18
- DOI
- https://doi.org/10.1109/tpami.2026.3754492
- Primary Topic
- Sparse and Compressive Sensing Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00