Using Machine Learning to identify the social determinants of the mid-life decline in mental health

Abstract Background The mid-life decline in mental well-being, depicted as a U-shaped pattern, is debated across disciplines. This study seeks to identify the social determinants of this decline and the overall contribution of social determinants to changes in mid-life mental health. Methods We use data from the 1970 British Cohort Study, from the ages 26, 34, and 42 (N = 6992, 51.5% female). Mental distress was measured using the 9-score Malaise Inventory. Mental distress increased from age 34 (M = 1.67) to 42 (M = 1.86; p < 0.001), with 17.8% declining by at least 1 SD on the inventory. We contrasted this group of decliners with those exhibiting stable mental health. We included socioeconomic and family factors, as well as physical health and health behaviours as potential determinants of mental health decline (at ages 26, 34, or at birth). We combine Random Forest and logistic regressions. Results The Random Forest’s Variable Importance (VI; root Mean Squared Error (MSE)), revealed social class at birth (VI = 0.023), physical multimorbidity at age 26 (VI = 0.22), and income quintile at age 34 (VI = 0.022) as most important predictors. Regressions confirmed that low birth social class (OR = 1.51 [1.25-1.83]), high-income at age 34 (OR = 0.68 [0.53-0.88]), and physical multimorbidity at age 26 (OR = 1.25 [1.06-1.48]) predict mental health decline. For those with high income and social class, the predicted probability of mid-life mental health decline is 0.109 (CI = [0.08-0.139]), rising to 0.277 (CI = [0.227-0.324]) for high risk individuals. Social class does not predict changes in mental health between ages 26 - 34. Interpretation We show that various social determinants of health contribute to the observed decline in mental health during mid-life. Social class at birth, income, and physical health are the most relevant predictors; they reduce the risk of decline from up to 27% to 11%. The contribution of social class appears specific to mid-life, and it could help explain the U-shaped pattern. Key messages • Social Factors Matter: Social class, income, and health predict mid-life mental decline. • Mid-life Class Impact: Social class uniquely affects mental health decline in mid-life.