A data re‐analysis confirms the value of symptom networks in predicting psychotherapy outcome

In a paper published in this journal1, we reported that person-specific symptom associations were highly predictive of short- and long-term outcome of psychotherapy for patients with chronic depression. In that analysis, individual symptom associations and their change through treatment were indicated by person-specific symptom networks, estimated by a multilevel longitudinal network model. Subsequently, the estimated change in symptom associations was used to predict clinician-rated treatment outcome. The network model that we used assumed that all change in symptom severity during psychotherapy was due to changing symptom associations. In contrast, traditionally used models in psychotherapy research consider that, in addition to the variables of interest (e.g., treatment group), an unknown number of unmeasured variables (e.g., developing therapeutic alliance) might impact change in symptom severity. Traditional models capture the collective effect of these unmeasured variables by a main effect of time2. By assuming that change in symptom severity is only due to changing symptom associations – i.e., not including the main effect of time – our network model did not consider these further potentially influencing factors. Moreover, when predicting short- and long-term treatment outcome, only changes in symptom associations during treatment, but not changes in symptom severity, were included as predictors. In other terms, our study did not assess to what extent change in symptom associations show an improved prediction of (long-term) treatment outcome beyond change in symptom severity. Aligning the analysis with the more traditional framework of assessing treatment effects, i.e. changes in symptom severity, ensures that findings can be more directly compared and understood by clinical researchers and clinicians, alongside the broader psychotherapy literature. Additionally, it can clarify the incremental predictive value of symptom associations. We therefore extended our analysis considerably, in order to allow a comparison with more traditional treatment effect models. The network model was extended by including a linear main effect of time and the interaction between time and treatment group. This extended network model considers that change in symptom severity might be due to changes in symptom associations and/or time (implicitly capturing the effect of unmeasured variables). We estimated this network model with the original data on self-rated depressive symptoms during psychotherapeutic treatment for chronic depression. We then re-ran the outcome prediction analysis to determine the predictive value of the updated symptom association change parameters for short- and long-term treatment outcome. Going beyond the original paper, in this further analysis, we also evaluated the predictive performance of symptom severity and symptom associations, and to what extent symptom associations predict beyond symptom severity. In this re-analysis, the unique prediction with change in symptom associations (based on the updated network model) yielded results similar to the original findings1, explaining 53% of the variance of depression severity at treatment end (adjusted R2=0.53; 95% credibility interval, CI: 0.47-0.58). Change in symptom severity during treatment explained 47% of the variance (adjusted R2=0.47, 95% CI: 0.40-0.53). The combined prediction with change in symptom severity and change in symptom associations increased the explained variance to 56% (adjusted R2=0.56, 95% CI: 0.50-0.61). For observer-rated depression severity one year after treatment, change in symptom associations (based on the updated network model) explained 51% of the variance (adjusted R2=0.51, 95% CI: 0.45-0.56), similar to the original results. A third of the variance (32%) could be uniquely explained by change in symptom severity during therapy (adjusted R2=0.32, 95% CI: 0.24-0.39). The combined prediction with change in symptom severity and change in symptom associations increased the explained variance to 54% (adjusted R2=0.54, 95% CI: 0.48-0.58). Two years after treatment, the predictive performance was 24% for change in symptom associations alone (adjusted R2=0.24, 95% CI: 0.16-0.32), 22% for change in symptom severity alone (adjusted R2=0.22, 95% CI: 0.13-0.29), and 25% for both variables (adjusted R2=0.25, 95% CI: 0.17-0.33). Thus, we found that, even when the network model accounted for the impact of unmeasured variables (by including the main effect of time), symptom associations remained highly predictive of short- and long-term treatment outcome. This suggests that the outcome prediction with symptom associations found in the original study was not impacted by the modeling choice made during the person-specific network estimation. The fact that the variance explained by both symptom severity and symptom associations is similar to the variance explained by each of these predictors individually implies that they are highly related. In other terms, the unique and joint predictive value of change in symptom severity and change in symptom associations overlap to some extent, and parts of the variance in treatment outcome was explained by both. While this might be due to a statistical artefact, it could also indicate a serial dependence of these effects, according to which changes in symptom associations might cause change in symptom severity (as proposed by the network theory3) or vice versa. Most importantly, we found substantial incremental predictive value for symptom associations beyond predictions with symptom severity (9% at the end of treatment, 22% one year later, and 3% after two years). The incremental value of symptom associations, especially one year after treatment, indicates that they provide additional information about the treatment effect that is not captured by changes in symptom severity change alone. These findings further support the notion that symptom networks might offer new opportunities for a better understanding of the effects of psychotherapeutic treatment. Taken together, the findings of this re-analysis relate changes in symptom associations during psychotherapy to traditional models of change in symptom severity, and underscore the added value of the network approach for understanding treatment effects. From an empirical perspective, the substantial incremental predictive value of symptom network parameters can be considered promising. In addition to a possible empirical advantage over traditional models, network models offer another important benefit: a coherent theory of psychopathology and a novel theoretical approach for understanding treatment effects. Investigating changes in symptom associations could offer new clues into understanding how treatment affects the problem-maintaining mechanisms of mutually reinforcing symptoms. Symptom networks can also indicate the stability of certain mental health states, i.e. whether a person is likely to change between high and low symptom severity4, 5. The stability of high symptom severity at treatment start could provide crucial information on the readiness to change, and the stability of low symptom severity at treatment end might indicate vulnerability to relapse, thus being related to long-term treatment outcomes. Thus, changing symptom networks might reveal possible working mechanisms of mental health treatments, explain individual differences in treatment effects, and offer new avenues for improving current psychotherapeutic treatments.