Kappa Encoding Using the Huber Loss Function

The Kappa Regression algorithm is a previously proposed method with excellent performance but requires significant computational effort for parameter tuning. In the Feature Encoding process the distance, for a given feature, between two samples is computed in terms of a common unit, namely that of the tagged value. However, individual features may have outliers that need to be taken into account. In the parameter tuning process, Mean Square Error (MSE) or Mean Absolute Error (MAE) have traditionally been used as the loss function when performing encoding for each feature but neither may be the most appropriate so we instead investigate using Huber Loss (of which MSE and MAE are special cases). We can determine the Huber Loss parameter value that optimizes the overall performance. This approach ensures that the model remains robust to both outliers and substantial noise. Note that Huber Loss is used for feature encoding and the feature distances are then used to obtain sample distances (e.g, using the Euclidean Distance Metric). Performance is then evaluated using MAE. We demonstrate the benefit of this approach over a wide variety of datasets. In addition, we use a synthetic dataset to illustrate the effects of noise and outliers in the parameter-tuning process.