Respiratory Sounds Classification by Fusing the Time-Domain and 2D Spectral Features

Automated respiratory sounds classification is a challenging task. Currently, the most commonly used feature extraction methods in the respiratory sounds classification are based on two-dimensional time-frequency representation, which reflect more information in the frequency domain but usually ignore the key information in the time domain of audio signals. To overcome this limitation, we proposed a novel respiratory sounds classification method that integrates the time-domain and two-dimensional time-frequency representation features of audio signals. Specifically, we first used a Mel encoder to extract the spectral features from the Mel-spectrum and used time domain feature extraction module to capture the time domain features of the original audio signal respectively. Then, we designed co-attention module for efficiently fusing these two different types of features. Finally, we inputted the fused features into the classifier for accurate classification of respiratory sounds. Experimental results on the ICBHI 2017 and SPRSound datasets demonstrate that our method achieved state-of-the-art performance. Specifically, it surpassed previous best results by 1.00% on ICBHI 2017 dataset and exceeded the best results in the four subtasks of SPRSound dataset by 3.01%, 4.89%, 5.70%, and 3.87%, respectively.