Voice-Image Cross-Modal Human Fatigue Detection Based on CNN-ELM Hybrid Model

The deepening of human fatigue will lead to the reduction of exercise ability and work efficiency, the increase of errors and accidents, and even the occurrence of organic diseases. Obviously, it is significant to understand the impact of human fatigue on the health, safe production and safe work of different people. At present, fatigue detection is mostly carried out through EEG and EMG signals. These methods usually have the disadvantages of contact and non-realtime. In response to the aforementioned issues in the process of human fatigue detection, this article effectively applies the visual image analysis method of spectrograms to human fatigue detection and proposes a cross-modal human fatigue detection method based on speech spectral image recognition. First, Mel spectrograms of speech segments in the corpus are extracted, and a fatigue spectrogram data set is established. A deep learning model is established through convolutional neural network (CNN) and extreme learning machine (ELM) for spectral image recognition and fatigue detection. CNN is used to extract features from the input image. The feature mapping will eventually be encoded into a one-dimensional vector and sent to ELM for classification. The experimental results indicate that the speech spectrum image features extracted by this method have better fatigue characterization ability than traditional acoustic features.