Comparison of Ordinal Classification Frameworks: An Empirical Benchmark of Performance and Efficiency

Purpose: This research systematically compares three ordinal classification frameworks (FH, DRM, CS) using tree-based ensemble algorithms (RF, XGB, CAT). It primarily aims to evaluate their predictive performance and computational time, addressing the relative lack of empirical guidelines for ordinal tabular data.Methods: Twelve model combinations were evaluated across twelve diverse benchmark datasets categorized by class cardinality, focusing primarily on nine ordinal models. In order to ensure statistical reliability, experiments were repeated over 30 independent trials using stratified sampling, with hyper parameter optimization via grid search. Predictive performance was assessed using the QWK metric to strictly penalize ordinal misclassifications, while computational efficiency was gauged by training time.Results: Predictive performance depended on class cardinality. Performance differences were insignificant in low-cardinality settings (3–4 classes, p=0.1434) but significant in high-cardinality settings (5–7 classes, p=0.0010), where DRM_XGB ranked best (mean rank 2.0000). Conversely, training speed differed significantly across all datasets (p<0.0001); DRM_XGB was the fastest (overall mean rank 1.0000).Conclusion: The findings highlight that selecting the optimal 'Framework-Algorithm synergy' is critical for ordinal classification. The DRM_XGB combination is highly recommended for high-cardinality settings, while either DRM or CS can be flexibly deployed in low-cardinality environments depending on computational constraints. This study clarifies the structural and computational boundaries of ordinal approaches, providing evidence-based guidelines for designing robust models for ordinal tabular datasets.