Catching the Blackdog Easily: A Convenient Depression Diagnosis Method Based on Audio-Visual Deep Learning

Depression has currently become a serious social problem worldwide. However, the need for experienced doctors and tedious medical examinations greatly increases the inconvenience in diagnosing the depression. A convenient depression diagnosis method can significantly improve the medical experience of depression patients, and can greatly reduce the workload of doctors. In this paper, a Convenient Depression Diagnosis method based on Audio-Visual Deep Learning (CDD-AVDL) is proposed. CDD-AVDL exploits the videos of testers reading a specially-designed text, and note that the videos contain many subconscious human reactions (e.g., micro expressions, voice variations), which are difficultly affected by the artificial interventions, thus enabling the depression diagnosis results more accurate. In CDD-AVDL, the source features are first extracted from audios and visuals, and then the time-sequential features are extracted. Finally, a full connection layer and a convolution layer fusion are responsible for fusing the audio-visual features to yield the depression probabilities. Extensive experiments and clinical tests show that CDD-AVDL outperforms the state-of-the-arts in terms of the accuracy of depression diagnosis. Moreover, the data collection manner in CDD-AVDL is convenient, and the training cost of CDD-AVDL is very low.