Exploring the Transitivity Assumption in Network Meta‐Analysis: A Novel Approach and Its Implications

The feasibility of network meta-analysis depends on several factors, one of which is the validity of the transitivity assumption that posits no systematic differences in the distribution of effect modifiers across treatment comparisons within a connected network. However, evaluating transitivity is complex for relying on epidemiological grounds. Therefore, establishing a methodological framework to evaluate this assumption is challenging. We propose a novel approach, which involves calculating dissimilarities between treatment comparisons based on study-level aggregate participant and methodological characteristics reported across studies and applying hierarchical clustering to cluster similar comparisons. This approach detects "hot spots" of potential intransitivity in the network, enabling empirical exploration of transitivity and semi-objective judgments. Our approach quantifies clinical and methodological (non-statistical) heterogeneity within and between treatment comparisons by computing the dissimilarities across studies in key characteristics acting as effect modifiers. The investigated networks showed varying between-comparison dissimilarities, indicating variability in the clinical and methodological heterogeneity of the networks. Several pairs of treatment comparisons with "likely concerning" non-statistical heterogeneity were identified, and some studies were organized into several clusters, suggesting potential intransitivity in the networks. These findings necessitate a closer examination of the evidence base, and such scrutiny becomes pivotal in determining whether concerns about the feasibility of network meta-analysis are justified. Similar to statistical heterogeneity, heterogeneity in clinical and methodological characteristics of the collected studies should be expected and appropriately assessed. Our proposed approach facilitates the evaluation of transitivity using well-established methods and can be applied to newly planned and published systematic reviews.