Exploring biases related to the use of large language models in a multilingual depression corpus
Recent advancements in Large Language Models (LLMs) present promising opportunities for applying these technologies to aid the detection and monitoring of Major Depressive Disorder. However, demographic biases in LLMs may present challenges in the extraction of key information, where concerns persist about whether these models perform equally well across diverse populations. This study investigates how demographic factors, specifically age and gender affect the performance of LLMs in classifying depression symptom severity across multilingual datasets. By systematically balancing and evaluating datasets in English, Spanish, and Dutch, we aim to uncover performance disparities linked to demographic representation and linguistic diversity. The findings from this work can directly inform the design and deployment of more equitable LLM-based screening systems. Gender had varying effects across models, whereas age consistently produced more pronounced differences in performance. Additionally, model accuracy varied noticeably across languages. This study emphasizes the need to incorporate demographic-aware models in health-related analyses. It raises awareness of the biases that may affect their application in mental health and suggests further research on methods to mitigate these biases and enhance model generalization.
Authors
- Matthew H. Hotopf (ORCID: https://orcid.org/0000-0002-3980-4466)
- Sara Siddi (ORCID: https://orcid.org/0000-0002-1494-9028)
- Juan Rafael Orozco‐Arroyave (ORCID: https://orcid.org/0000-0002-8507-0782)
- Yuezhou Zhang (ORCID: https://orcid.org/0000-0003-4817-3835)
- Tomás Arias‐Vergara (ORCID: https://orcid.org/0000-0001-9405-4154)
- Srinivasan Vairavan (ORCID: https://orcid.org/0000-0001-7975-8436)
- Til Wykes (ORCID: https://orcid.org/0000-0002-5881-8003)
- Josep María Haro (ORCID: https://orcid.org/0000-0002-3984-277X)
- Faith Matcham (ORCID: https://orcid.org/0000-0002-4055-904X)
- Elmar Nöth (ORCID: https://orcid.org/0000-0002-3396-555X)
- Brenda W.J.H. Penninx (ORCID: https://orcid.org/0000-0001-7779-9672)
- Amos Folarin (ORCID: https://orcid.org/0000-0002-0333-1927)
- Judith Dineley (ORCID: https://orcid.org/0000-0001-5541-6853)
- Richard Dobson (ORCID: https://orcid.org/0000-0003-4224-9245)
- Vaibhav A. Narayan (ORCID: https://orcid.org/0000-0001-6092-5150)
- Femke Lamers (ORCID: https://orcid.org/0000-0003-4344-5766)
- Paula Andrea Pérez-Toro (ORCID: https://orcid.org/0000-0002-2727-2116)
- Nicholas Cummins (ORCID: https://orcid.org/0000-0002-1178-917X)
- Andreas Maier (ORCID: https://orcid.org/0000-0002-9550-5284)
- Raquel Iniesta
Institutions
- University of Sussex (GB)
- Friedrich-Alexander-Universität Erlangen-Nürnberg (DE)
- King's College London (GB)
- South London and Maudsley NHS Foundation Trust (GB)
- Universidad de Antioquia (CO)
- Alzheimer’s Disease Neuroimaging Initiative (US)
- Amsterdam Neuroscience (NL)
- Parc Sanitari Sant Joan de Déu (ES)
- Cummins (United Kingdom) (GB)
- Public Health Service of Amsterdam (NL)
- Janssen (United States) (US)
- University College London (GB)
Publication Details
- Journal
- Scientific Reports
- Published
- 2025-10-16
- DOI
- https://doi.org/10.1038/s41598-025-19980-x
- Citations
- 4
- Primary Topic
- Mental Health via Writing
- Type
- article
- Field-Weighted Citation Impact
- 9.75
Funders
- European Federation of Pharmaceutical Industries and Associations
- National Institute for Health and Care Research
- Department of Health and Social Care
- King's College London
- Universidad de Antioquia
- NIHR Maudsley Biomedical Research Centre