Machine learning and natural language processing for the identification of potential mental disorders among school-age children: a prospective birth cohort study

Early identification of childhood mental health disorders is a critical public health objective. Existing screening approaches, largely dependent on observer reports, are resource-intensive and may overlook subtle internalized symptoms. The analysis of children’s linguistic expression presents a scalable and potentially more objective alternative. This study evaluates whether combining natural language processing (NLP) of children’s essays with conventional risk factors improves the detection of mental health difficulties in school-age populations, relative to models based on a single data source. We conducted a prospective analysis using data from the UK-based National Child Development Study (NCDS), a national birth cohort initiated in 1958. Data from birth, age 7, and age 11 assessments were analyzed. The final sample included 8,981 children (4,428 [49.3%] female) who completed a creative writing essay at age 11 describing their imagined life at age 25. Predictors comprised traditional risk factors (perinatal, socioeconomic, and parental engagement variables) and linguistic features computationally extracted from the essays. The primary outcome was potential mental health disorder at age 11, defined as scoring above the 95th or 90th percentile on the teacher-completed Bristol Social Adjustment Guide (BSAG). The mother-completed Rutter A Scale was used for sensitivity analysis. Machine learning models incorporating various predictor combinations were developed, and their predictive performance was evaluated using area under the receiver operating characteristic (AUROC) values. Using BSAG 95th percentile threshold, models combining top five selected variables with essay features achieved significantly higher predictive capability (AUROC:0.77, 95%CI:0.71–0.83) compared to models using all variables (AUROC:0.70, 95%CI:0.63–0.76) or essay features alone (AUROC:0.67, 95%CI:0.60–0.74). At 90th percentile threshold, this integrated approach showed similar improvement (AUROC:0.81, 95%CI:0.78–0.85). Key predictors included gestational length, maternal parity, parental age, residential characteristics, parental engagement metrics, and children’s body mass index. Sensitivity analyses using Rutter A Scale confirmed these findings. In this prospective birth cohort study, integrating NLP analysis of children’s essays with a small set of key risk factors substantially improved the identification of potential mental health disorders. This integrated approach represents a potential paradigm for developing scalable, objective screening tools, but requires validation in contemporary, diverse pediatric populations before clinical consideration.