CALA-Net: Intelligent depression detection with multimodal fusion and Tree-of-Thoughts framework

Depression is recognized as the second leading cause of global disease burden, yet efficient and precise detection methods remain underexplored. While existing approaches have made progress in identifying depressive symptoms, they are limited by insufficient multimodal semantic analysis, subjective assessment bias, and inadequate detection performance. This study aims to address these gaps by proposing the Cross-Attention and LLM-Assisted Network (CALA-Net), a multimodal framework for depression screening. The framework integrates a Temporal-Spatial Cross-Attention (TSCA) module to fuse audio and facial features, employs a fine-tuned large language model (LLM), and utilizes a Tree of Thoughts (ToT) prompting mechanism to systematically evaluate participants’ dialogue or self-report texts along standardized psychological feature dimensions. The extracted multimodal features are then processed by a random forest classifier for final depression assessment. Evaluation was conducted on the publicly available D-Vlog and LMVD datasets, as well as the internally developed FJSI structured clinical interview dataset, demonstrating the model’s capability across different evaluation settings. The TSCA module achieved F1-Scores ranging from 0.740 to 0.787 across all three datasets. The complete CALA-Net framework, incorporating the text modality, showed improved performance on the FJSI dataset with F1-Scores between 0.826 and 0.841. These findings provide valuable insights into multimodal depression detection and present a novel approach to support objective, automated screening for depression in clinical settings. • We propose CALA-Net, a multimodal framework for automated depression screening. • We designed a TSCA module for audio and facial features fusion via temporal-spatial cross-attention. • We found the combination of audio, facial, and textual modalities can improve depression detection. • CALA-Net achieves F1-Scores of 0.826-0.841 on clinical interview data.