Tackling No-show Imbalance problems for Healthcare Appointment datasets

Medical appointment no-shows have a significant impact on the revenue, cost and resource utilization for almost all healthcare systems. To address this problem, numerous efforts have been made in the past to apply machine learning algorithms for predicting patient no-shows. However, the issue of class imbalance that often arises in these cases has largely been neglected. Given the highly imbalanced nature of our data, we propose a novel ensemble method for classification that generates an arbitrary number of balanced splits of the data. This method uses Instance Hardness as a weighting mechanism to create balanced bags, with each bag containing all minority instances and a filtered subset of majority instances. We then employ the Histogram-based Gradient Boosting classifier as the base learner for each bag. This approach allows the base learners to train on different balanced bags, each reflecting varied characteristics of the training data. We tested our proposed method on four no-show datasets, and the results demonstrate a substantial improvement in classification performance compared to other methods.