Variational oblique predictive clustering trees

• Introduces VSPYCT, a variational Bayesian oblique decision tree for struc- tured output prediction. • Captures parameter uncertainty through variational inference in each tree split. • Matches or outperforms SPYCT ensembles on classification and multi-target regression tasks. • Provides feature importance scores and visual interpretability within a single-tree framework. • Demonstrates robustness to spurious features and performs well across varied dataset properties. Oblique predictive clustering trees (SPYCTs) are semi-supervised multi-target prediction models mainly used for structured output prediction (SOP) problems. They are computationally efficient and when combined in ensembles they achieve state-of-the-art results. However, one major issue is that it is challenging to interpret an ensemble of SPYCTs without the use of a model-agnostic method. We propose variational oblique predictive clustering trees, which address this challenge. The parameters of each split node are treated as random variables, described with a probability distribution, and they are learned through the Variational Bayes method. We evaluate the model on several benchmark datasets of different sizes. The experimental analyses show that a single variational oblique predictive clustering tree (VSPYCT) achieves competitive, and sometimes better predictive performance than the ensemble of standard SPYCTs. We also present a method for extracting feature importance scores from the model. Finally, we present a method to visually interpret the model’s decision making process through analysis of the relative feature importance in each split node.