Prediktion av Behandlingsmisslyckande och Avhopp i en Online Intervention för Alkoholbrukssyndrom : Användning av Maskininlärning och Naturlig Språkbehandling för Prediktion av Behandlingsutfall

Alcohol Use Disorder (AUD), a clinical diagnosis defined in the DSM-5 by 11 criteria that span harmful use and dependence (see Section 2.1), is a global problem. In 2019 alone, alcohol consumption contributed to around 2.6 million deaths (4.7% of all deaths globally), and over 400 million people lived with alcohol use disorders. In the last 20 years, Internet-delivered Cognitive Behavioral Therapy (ICBT) has emerged as an effective treatment for AUD. However, the public health impact of ICBT is limited by high rates of non-response and dropout. A promising solution to this problem is the use of predictive models to identify patients at risk before treatment begins, enabling adaptive treatment strategies that can lead to improved treatment outcomes. Unfortunately, previous research in this field suggests that these models typically show moderate performance, with AUC scores in the 0.60-0.70 range. Prior research relies on structured data, such as severity ratings (e.g., the number of standard drinks). An advantage of ICBT is that unstructured text data is automatically collected, but there has not been much work on using unstructured text data for prediction purposes. This thesis investigated whether the use of text from motivational interviewing (MI), a novel data source in this context, could improve the prediction of non-response and dropout in an ICBT intervention for AUD. The data used in this study comes from a randomized controlled trial of ICBT for AUD. We compared the effects of various feature sets on performance using nested cross-validation and took steps to prevent data leakage. Logistic Regression (LR) and XGBoost models were trained on baseline characteristics (e.g., questionnaire data and demographics), text-derived features from topic modeling, and a combination of both. Additionally, a cutting-edge approach based on few-shot fine-tuning of Sentence Transformers (SetFit) for text classification was used to predict treatment outcomes using the raw text data from MI. Lastly, we trained LR and XGBoost on a combination of baseline characteristics and UMAP-reduced SetFit embeddings. The results show that the MI text, whether we trained models on text-derived features or used the SetFit framework, exhibited no predictive power, with performance metrics hovering around the chance level. Instead, baseline characteristics were the most predictive of treatment outcomes. The simpler LR model consistently outperformed the more complex XGBoost model, achieving an AUC of 0.705 for non-response and 0.648 for dropout using baseline data. Combining baseline data with text-derived features led to marginal and inconsistent improvements. Consistent with previous research, the findings suggest that current measurement practices in clinical psychology and psychiatry, which focus on collecting baseline characteristics, yield moderate performance (AUC 0.60-0.70) when treatment outcomes are 12 weeks away. The findings also show that the text from MI used in this study does not contain enough predictive information to improve upon this, regardless of model complexity. The primary conclusion of this study is that the key to improving the performance of predictive models in clinical psychology and psychiatry lies in collecting and training models on data with greater predictive power rather than relying solely on baseline characteristics or developing more sophisticated machine learning models. Future research could investigate how dynamic in-treatment data, which captures patients’ behaviors and experiences in real time, impacts performance.