On the Determination of a Suitable Distance Metric for the Kappa Regression Algorithm

The Kappa Regression algorithm is a robust approach that first converts the distance between a single feature of two samples into a common metric, namely that of the tagged value. This can be done for both categorical and numerical features. Given the distance for each feature for a given pair of samples, one can then combine these to obtain the distance between the pair of samples using various distance metrics such as Euclidean and Manhattan distance. The choice of distance metric can significantly influence how models interpret the similarity between data points. Therefore, when selecting a distance metric, it is important to consider factors such as feature sensitivity, outlier sensitivity, dimensionality, and model interpretability. We use the Minkowski distance metric, characterized by a parameter controlling the distance order, and investigate its effect on performance. This approach allows the model to be robust to outliers while maintaining sensitivity to significant distance variations. Mean Absolute Error was used as the performance metric and a wide variety of datasets were considered in demonstrating the value of using Minkowski distance for the kappa regression algorithm. We also compare it with other Regressors that have been shown to benefit from using the Minkowski Distance Metric.