Real-World Spectral Compressive Imaging by Learning a Physics-Informed Forward Model

Coded aperture snapshot spectral imaging (CASSI) enables snapshot acquisition of hyperspectral image (HSI) by compressing a three-dimensional spectral data into a single two-dimensional measurement. While recent learning-based reconstruction methods achieve impressive performance on simulated datasets, their performance often degrades significantly on real CASSI systems, revealing a simulation-to-reality gap. This gap arises from a mismatch between the complex real imaging process and the reconstruction mappings learned from idealized simulated data. To bridge this simulation-to-reality gap, we propose a learnable physics-informed forward model (LPFM) that incorporates the physical structure of CASSI into a learnable proxy network to model the complex real imaging process. To train the LPFM, we construct the first real-world spectral compressive imaging (rSCI) dataset. The rSCI dataset contains 219 pairs of static HSI-CASSI images across diverse indoor and outdoor scenes, as well as additional dynamic CASSI videos. Leveraging rSCI and LPFM, we establish the first real-world benchmark for CASSI reconstruction. Extensive experiments demonstrate that incorporating data generated by LPFM effectively reduces the simulation-to-reality gap and consistently improves reconstruction performance across various state-of-the-art methods.

Authors

Institutions

Publication Details

Journal
IEEE Transactions on Pattern Analysis and Machine Intelligence
Published
2026-11-18
DOI
https://doi.org/10.1109/tpami.2026.3754492
Primary Topic
Sparse and Compressive Sensing Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Real-World Spectral Compressive Imaging by Learning a Physics-Informed Forward Model

Xiangtian Ma, Lingqin Kong, 宋维涛, Xin Wang et al.
IEEE Transactions on Pattern Analysis and Machine Intelligence
Sparse and Compressive Sensing Techniques
article

Real-World Spectral Compressive Imaging by Learning a Physics-Informed Forward Model

Xiangtian Ma, Lingqin Kong, 宋维涛, Xin Wang, Yuchen Wang, Hua Huang, Lizhi Wang
article en

Abstract

Coded aperture snapshot spectral imaging (CASSI) enables snapshot acquisition of hyperspectral image (HSI) by compressing a three-dimensional spectral data into a single two-dimensional measurement. While recent learning-based reconstruction methods achieve impressive performance on simulated datasets, their performance often degrades significantly on real CASSI systems, revealing a simulation-to-reality gap. This gap arises from a mismatch between the complex real imaging process and the reconstruction mappings learned from idealized simulated data. To bridge this simulation-to-reality gap, we propose a learnable physics-informed forward model (LPFM) that incorporates the physical structure of CASSI into a learnable proxy network to model the complex real imaging process. To train the LPFM, we construct the first real-world spectral compressive imaging (rSCI) dataset. The rSCI dataset contains 219 pairs of static HSI-CASSI images across diverse indoor and outdoor scenes, as well as additional dynamic CASSI videos. Leveraging rSCI and LPFM, we establish the first real-world benchmark for CASSI reconstruction. Extensive experiments demonstrate that incorporating data generated by LPFM effectively reduces the simulation-to-reality gap and consistently improves reconstruction performance across various state-of-the-art methods.

IEEE Transactions on Pattern Analysis and Machine IntelligenceVol. PP
Beijing Institute of Technology (CN), Beijing Normal University (CN)
Sparse and Compressive Sensing Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon