Interpretable, multidimensional evaluation framework for causal discovery from observational i.i.d. data

Nonlinear causal discovery from observational data imposes strict identifiability assumptions on the formulation of structural equations utilized in the data-generating process. However, in real-life settings, the ground-truth mechanism responsible for cause-effect transformations is unknown. Thus, it is impossible to verify its identifiability. This is the first research to assess the performance of structure learning algorithms from seven different families in non-identifiable settings with an increasing degree of nonlinearity. The evaluation of structure learning methods under assumption violations requires a rigorous and interpretable approach that quantifies both the structural similarity of the estimation with the ground truth and the capacity of the discovered graphs to be used for causal inference. Motivated by the lack of a unified performance assessment indicator, we propose an interpretable, multidimensional evaluation framework, specifically tailored to the field of causal discovery from i.i.d. data. In particular, we introduce a six-dimensional evaluation metric, called distance to the optimal solution, which aims at providing a holistic overview of the performance of structure learning techniques. Our large-scale simulation study, which incorporates seven experimental factors, shows that hybrid Bayesian networks outperform most recently introduced continuous optimization techniques under certain conditions. Additionally, causal order-based methods yield results with comparatively high proximity to the optimal solution. • Our framework evaluates 14 causal discovery models in non-identifiable settings. • Hybrid Bayesian networks outperform most continuous optimization models. • Causal order-based structure learning achieves the current SOTA performance.