Adapting performance metrics for ordinal classification to interval scale: length matters

Abstract In the field of supervised machine learning, accurate evaluation of classification models is a critical factor for assessing their performance and guiding model selection. This paper delves into the domain of ordinal classification and raises the question of adapting ordinal metrics to the interval scale. In scenarios where measurements are recorded at intervals, not only the order but also their length assume significance, and this promotes the adoption of novel performance metrics. Initially, we revisit two existing confusion matrix-based ordinal metrics and introduce a normalization technique to render them comparable and enhance their practical utility. We extend our focus to classification by intervals, proposing a robust framework for adapting ordinal metrics to the interval scale, and applying it to the aforementioned ordinal metrics. We address the challenge of unbounded rightmost intervals, a common issue in practical applications, from both theoretical and simulation perspectives, by providing a solution that enhances the applicability of the proposed metrics. To further explore practical implications, we conducted experiments on real-world datasets. The results reveal a promising trend in the use of interval-scale metrics to guide hyper-parameter tuning for improving model performance.