Do We Really Need Hundreds of Machine Learning Models in Industry?

Abstract Recent practice in tabular data-driven applications in the petroleum industry often follows the classic machine learning workflow, which selects the so-called "best" model based solely on accuracy. This accuracy-focused workflow involves exhaustive model comparison across hundreds of algorithms and tends to favor complex, hybrid or specific-tailored models. As a result, models that are well suited for field deployment, such as decision trees, are frequently overlooked despite offering lightweight structures and essential interpretability that support operator trust and practical use. In this regard, this study challenges the traditional paradigm by proposing a competitive oblique decision tree that strikes an optimal balance between accuracy, interpretability, and lightweight model structure. The proposed model reformulates decision tree training problem, which involves learning optimal nested IF–THEN prediction rules, as an unconstrained optimization task. A soft approximation is applied to enable differentiability. This reformulation enables effective optimization of all prediction rules through our gradient-based entire tree optimization approach. Extensive experiments on 26 tabular datasets show that our model achieves competitive predictive accuracy compared to 26 commonly used machine learning algorithms. Our tree with constant prediction, on regression datasets achieves an average improvement of 6.25% over traditional decision tree (CART), while our tree with linear prediction yields 7.22% improvement over CART and slightly outperforms random forest with a 0.1% gain. Beyond accuracy, our tree with linear prediction features a lightweight structure with only 4,650 parameters, several orders of magnitude smaller than random forest with 0.96 million, resulting in a 71 times speedup in prediction time. Interpretability is preserved through concise IF–THEN rules, with about 24.37≈ 21 prediction rules far fewer than the approximate 212.47 ≈ 5763 rules typically generated by CART. These performance advantages, combined with open-source availability, make our tree a strong alternative to hundreds of machine learning models and eliminates the need for tedious model selection.