Continuous ranked probability score and collaborative-filtering-based sampling recommendation for high product-quality-level prediction

The imbalance inherent to product defect data hinders product-quality-level prediction, and many sampling methods have been proposed to address this imbalance. The performance of each sampling method varies depending on dataset characteristics. However, existing studies have primarily relied on simple meta-features when recommending sampling methods, which results in suboptimal predictive performance. This paper proposes the continuous ranked probability score and collaborative-filtering-based sampling recommendation (CRPS-CFSR) method that emphasizes differences in the distribution of mined data to recommend better sampling methods for new datasets. First, an improved dataset similarity metric that incorporates a feature importance explanation model and measures the distance between data distributions, CRPS2, is developed. It accommodates the non-normal product-quality data that are common in practice. Second, a collaborative filtering algorithm is combined with the CRPS2 metric to recommend a sampling method for a target dataset. It tests the effectiveness of current sampling methods in classifying quality levels using historical datasets, then utilizes sampling method performance and data similarity metrics to make recommendations. Third, the proposed CRPS-CFSR method is evaluated on 12 product-quality datasets from the UCI repository, outperforming existing benchmark algorithms and matching the optimal sampling method with a higher probability.