Speech Depression Recognition from the Selfreference Effect Using LSTM with ResNet

Deep neural networks (DNNs) have recently achieved great success in various learning tasks and have also been used for the classification of depression. While DNNs are showing their potential in the classification task, they cannot fully utilize the temporal information. To mine the relevance of signals in audios and increase the diversity of information, a neural network architecture for the purpose of using sequential information is proposed in this paper. The proposed structure is composed of two separated lower networks and one upper network. We refer to these as LSTM layers, Residual Networks (ResNet) layers, and connected layers, respectively. The LSTM layers can learn long-term dependencies from the learned local features. The ResNet layers learn the spectro-temporal locality from spectrogram images. Finally, the connected layers summarize the outputs of two networks to take advantage of the complementary features of the LSTM and ResNet by combining them. In addition to this, we experiment on our own designed dataset, which was designed based on the Self-reference Effect (SRE) experimental paradigm of psychology. Our results show the proposed combination structure can improve the accuracy of speech depression recognition.