Spectral pixels as images: CNN-based pixel classification of 4D hyperspectral data for nerve and ligament differentiation

Hyperspectral imaging is actively gaining popularity for multiple biomedical applications due to its ability to capture rich information from the electromagnetic spectrum. In particular, it can be used to differentiate multiple tissues based on their fluorescence. The reliable differentiation of nerve and ligament tissues is of particular interest in the surgical setting since these tissues have similar appearance for the human vision. By exciting the tissue with several consecutive wavelength bands from the longwave ultraviolet range and registering the tissue response in the visible light range, we construct a 4D hyperspectral image. We present a very first investigation into tissue classification from 4D hyperspectral images through a novel approach that treats excitation–emission matrices of individual pixels as images and trains a deep convolutional neural network for their classification. We test our approach on four different datasets of nerve and ligament tissue excised from bovine trotters, with variable imaging conditions. We achieve an impressive 94–99% classification accuracy with only 1% of pixels used as training data. We further examine cross-dataset applicability of the trained deep neural models and their sensitivity to changes in imaging conditions. Finally, our approach is benchmarked against and is shown to be superior by 10.4%, on average, to established automated analysis techniques like linear unmixing for biomedical hyperspectral images.