DNet: a depression recognition network combining residual network and vision transformer

Depression is a prevalent and severe global mental disorder, yet its diagnosis and treatment encounter numerous challenges. This study introduces an efficient depression identification network, termed DNet. Our approach utilizes facial images and local facial images as crucial sources of data. Although people with depression may exhibit similar general facial expressions, subtle changes in specific facial areas (e.g., eyes, mouth) can be observed as the severity of depression increases. These changes are reflected in the models' attention mechanisms, highlighting the differences between severity levels. To achieve higher recognition accuracy, a method is required to fuse advanced semantic features between local and global features. Therefore, we propose DNet, comprising two key components: the Feature Extraction Module (FEM) and the Vision Transformer (ViT) Block. Specifically, FEM introduces an attention mechanism that considers both channel and positional information of the feature Map. Two FEMs are employed to separately process facial and local facial images, extracting critical features to generate highly semantic information-rich feature Maps. Subsequently, the feature maps of both images are concatenated along the channel dimension, Using FPN feature fusion, and the ViT Block is utilized to comprehensively learn advanced semantic features of local and global information related to different facial expression regions. Finally, a 1× 1 convolution layer and a fully connected layer are applied to adjust feature channels, yielding more robust predictive results and ultimately outputting depression prediction scores.We experimentally validate the DNet network on the AVEC2014 dataset and our self-constructed CZ2023 dataset, obtaining results of MAE = 6.09, RMSE = 7.85, and MAE = 6.73, RMSE = 8.47, respectively. These results affirm the effectiveness of the proposed method.