Balancing Fairness and Accuracy in Data-Restricted Binary Classification

Fair decision-making in Machine Learning (ML) remains a critical challenge, particularly when access to sensitive information is restricted due to legal, ethical, or organizational constraints. These limitations affect both accuracy and fairness, creating tradeoffs central to the deployment of ML systems in the real world. While prior work has studied fairness-accuracy tradeoffs, most approaches focus on model outputs rather than directly examining how restricted data access impacts fairness. This leaves an important gap: understanding how fairness constraints affect model performance under real-world data restrictions . To address this gap, we propose a framework that explicitly models fairness-accuracy tradeoffs in data-restricted environments. Unlike prior work, our approach analyzes the behavior of the optimal Bayesian classifier using a discrete approximation of the data distribution, allowing us to systematically isolate the effects of fairness constraints. We evaluate our framework on three benchmark datasets—Adult, Law, and Dutch Census—revealing key insights: (1) enforcing equal accuracy on imbalanced datasets can substantially degrade performance under additional fairness constraints, (2) individual and group fairness often impose conflicting constraints, and (3) decorrelating sensitive attributes from features does not usually reduce accuracy. These findings demonstrate that our framework provides an effective, structured approach for practitioners to assess fairness constraints in decision-making pipelines.