Coordinating high-resolution hyperspectral and RGB video acquisition of dynamic natural water scenes

A bimodal video imaging platform combining 371-band hyperspectral and red-green-blue (RGB) video acquisition systems was constructed and used to collect video imagery of the Lake Ontario shoreline at Hamlin Beach State Park in Rochester, New York, United States. We designed a video processing workflow to correlate video reflectance data collected by a line-scanning imaging spectrometer and a traditional RGB video camera for hyperspectral imagery prediction. Using the relationship between the hyperspectral video (HSV) data and RGB video, we tested our workflow by predicting hyperspectral image frames of dynamic natural water scenes from the RGB imagery at times prior to and following a time segment where we had developed a correlative model between the two imagery data streams. We acquired HSV using a Headwall Hyperspec micro-high efficiency visible and near-infrared imaging spectrometer in the low-rate video mode of our configuration and RGB data with a low-cost consumer GoPro Hero 8 Black. Hyperspectral image band predictions used distributions of absolute and normalized residuals in radiometrically calibrated reflectance spaces. Within visible wavelengths, 95% of the scene was predicted to within 2% absolute reflectance, which translates to ∼30% of signal level for water spectra. In the near-infrared regime, the normalized error percentage of the residuals sharply increased to ∼90% for 95% of the scene due to lack of band information from the RGB video imagery of our shallow water scene.