Modeling Harmonic Structure and Spectral Complexity in Speech for the Analysis of Depression and Emotional Expression

Major depressive disorder (MDD) is a common mental health condition that requires scalable and objective detection methods. Speech is a promising non-invasive digital biomarker, but many existing approaches rely on binary classification, lack interpretable acoustic modeling, and ignore demographic differences that affect speech patterns. This work proposes HEAR-Net, a unified speech-based framework for depression detection and emotion recognition that integrates acoustic features and demographic context. We propose a harmonic block to capture long-range harmonic patterns related to voice quality, a spectro-temporal orthogonal gating mechanism to model time and frequency information, and an entropy-aware pooling to reflect changes in expressive variability. We incorporate age and sex information to provide demographic context alongside acoustic features. Experiments on DAIC-WOZ and MODMA achieve average F1-scores of 0.93 and 0.92, with accuracies of 0.94 and 0.92, respectively. On CASIA and EmoDB, HEAR-Net reaches accuracies of 0.89 and 0.91 without external pre-trained models.