Two-channel image reconstruction from single-sensor data using optimized spatial light modulators

Many contemporary computer vision systems depend on multiple channels of spectral information within an image. Traditionally, these spectral channels are recorded either across distinct regions of a single sensor utilizing Bayer filters or through the deployment of separate sensors for each channel. In this work, we propose a novel imaging system that employs optimized modulators to individually optically encode two separate spectral bands of the incident light before recombining them on a single sensor. Knowledge of the modulation allows us to reconstruct two separate images from the single-channel data captured by the sensor. We present a framework that implements the entire image formation and reconstruction process using automatic differentiation techniques. This framework allows for the joint optimization of both the optical components and the reconstruction procedure, ultimately leading to enhanced imaging results. We validate the functionality of our proposed system through preliminary simulations results. Our findings suggest that this approach could be used to realize more compact imaging systems optimized for these multichannel applications in the future.