Raman spectral unmixing via multimodal time-frequency transformations and deep learning

Raman spectroscopy has been proved to have the potential to accurate diagnose a variety of diseases, and what we believe to be novel Raman probes or instruments for clinical applications were constantly developed. However, biological tissues are usually structurally complex. so that the Raman signals collected in vivo may come from a variety of chemical components, even different tissues. This work proposed a Raman spectral unmixing approach, which can separate the signals of different tissues from their mixed spectra. Specifically, multimodal frequency and time-frequency transformation were performed together to extract the different features of mixed spectra. An attention U-net model was introduced to predict the spectra of target tissues in each modality. Then multimodal fusion was conducted to filter and integrate effective information from the above modalities and obtained accurate unmixed spectra. Canine knee joints with osteoarthritis were selected as the research subject, and the spectra of subchondral bone and cartilage were successfully separated from their mixed spectra, which can be further applied in osteoarthritis research just like the actual measured spectra. This work will contribute to biological in vivo detection of Raman probes or instruments, enabling them to separate signals from different tissues, structures, and even biochemical molecular components, achieving more accurate prediction and diagnosis.