Development and validation of an LSTM model for predicting depressive symptoms in middle-aged and older adults: a prospective multi-cohort transcultural study

Abstract Background Depressive symptoms are common in middle-aged and older adults and pose challenges for early identification and follow-up. Longitudinal risk prediction may support earlier risk stratification by integrating repeated information across multiple life-course and health-related domains. We aimed to develop and validate a machine learning model to predict depressive symptoms in middle-aged and older adults, while identifying key predictive factors. Methods We developed an attention-based Long Short-Term Memory (LSTM) model using longitudinal data from the China Health and Retirement Longitudinal Study ( n = 8,036) across five waves. External validation was performed using data from the Health and Retirement Study ( n = 6,424). The model incorporated 28 predictors across sociodemographic, health-related, childhood experience, and intergenerational variables. The outcome was clinically significant depressive symptoms at the final wave. Performance was evaluated using Area Under the Receiver Operating Characteristic Curve (AUC), calibration curves, Brier score, threshold-dependent metrics and Decision Curve Analysis (DCA). We compared the LSTM with L2-regularized logistic regression, random forest, and XGBoost under the same framework. Shapley additive explanations (SHAP) were used for interpretability. Results The attention-based LSTM showed good discrimination, with an AUC of 0.807 (95% CI: 0.789–0.825) in the internal test set and 0.789 (95% CI: 0.774–0.805) in the external validation cohort, outperforming all the conventional non-sequential models in both cohorts. However, its discrimination was comparable to that of the plain LSTM without attention. Calibration plots showed generally acceptable agreement, particularly for the LSTM-based models, although some conventional models showed modest calibration deviations in the external validation cohort. Childhood relationships with parents, particularly mother-child relationships, emerged as the strongest predictor, followed by cognitive ability, self-rated memory, and employment status. Attention-weight plots indicated greater relative weighting of more recent survey waves. Conclusions This population-based study demonstrates that LSTM with attention mechanisms can effectively predict clinically significant depressive symptoms in aging populations across two national aging cohorts. Early parent–child relationships were among the most important predictive markers for late-life depressive symptoms.