Performance Evaluation for Detecting and Alleviating Biases in Predictive Machine Learning Models

Machine Learning (ML) is widely used in various domains but is susceptible to biases that can lead to unfair decisions. Bias can arise from biased data, algorithms, or data collection processes, making it crucial to develop methods that ensure fairness. This article introduces the Detect and Alleviate Bias (DAB) framework, a novel approach designed to identify and mitigate bias in ML models, focusing on sensitive attributes such as gender and race. The key contributions of DAB include: (1) a holistic pipeline that combines data pre-processing, model enhancement, situation testing, and bias mitigation techniques; (2) the application of Counterfactual Fairness testing to detect individual biases; and (3) the integration of multiple bias mitigation strategies to improve fairness in binary classification tasks. We demonstrate the practical implications of DAB through empirical experiments on two widely used datasets, showing that it reduces bias and improves fairness with a slight compromise in model performance, with a significant potential for making an impact in sensitive domains such as healthcare and criminal justice. The results show that pre-processing and post-processing mitigation techniques achieve the best improvements in fairness for unrepresentative datasets, while in-processing methods are more effective in enhancing fairness for representative datasets; however, an inverse relationship between model performance and fairness is observed.