Real-time Hyperspectral Imager with High Spatial-Spectral Resolution Enabled by Massively Parallel Neural Network

One-shot spectral imaging has been a hot research topic recently, with primary challenges in the efficient fabrication techniques of encoded masks and high-speed, high-accuracy algorithms for real-time imaging. We introduce a real-time hyperspectral imager that leverages multilayer thin film microfilters and the Massively Parallel Network (MP-Net). Each curved microfilter uniquely modulates incident light across the underlying 3 × 3 CMOS pixels, thereby rendering each pixel an efficient spectral encoder. MP-Net, specially designed to address transmittance variability and manufacturing errors such as misalignment and nonuniformities in thin film deposition, greatly increase the robustness to fabrication errors. A spectral resolution of 2.19 nm is achieved for monochromatic spectra. Tested in varied environments on both static and moving objects, the imager demonstrates high-fidelity spatial-spectral data reconstruction capabilities with a maximum imaging frame rate exceeding 30 fps. This hyperspectral imager represents a significant advancement in real-time, high-resolution spectral imaging, offering a versatile solution for applications ranging from remote sensing to consumer electronics.