Constructing multimodal counseling dataset for depressive state and feature analysis

In recent years, depression caused by stress and anxiety has been on the rise worldwide, and the number of people taking leave due to depression and other mental illnesses has been increasing in Japan as well. However, there is a serious shortage of counselors and psychiatrists, making early detection difficult. In this study, for early detection of depression, 25 Japanese university students were interviewed by a professional counselor for about 30 minutes, and a multimodal dataset consisting of text, audio, video, and heart rate data was constructed during the interview. Before and after the interviews, a questionnaire about the subjects’ current psychological state was also conducted. The constructed dataset was analyzed by extracting features using GiNZA, OpenSMILE, and OpenFace, and comparing them with the results of the questionnaire. In the future, we plan to conduct interviews with mentally ill subjects and collect data from them.