The Fairness Threshold Problem: A Governance Framework for Resolving Metric Conflicts in Enterprise AI Fairness Assessment
Enterprise AI fairness assessment frameworks require organisations to evaluate AI system outputs against defined fairness metrics. The three primary metrics — demographic parity, equalised odds, and counterfactual fairness — are well established in the technical literature and form the foundation of regulatory fairness expectations under the EU AI Act, Australian anti-discrimination law, and sector-specific guidance in financial services and healthcare. What the literature and regulatory frameworks do not resolve is what an organisation must do when a system satisfies one fairness metric but not another — a condition that the mathematical literature has demonstrated is the rule rather than the exception in real-world AI deployments. This paper identifies the conditions of the Fairness Threshold Problem and argues that it is not a technical problem but a governance design problem. The Fairness Threshold Problem arises from three structural properties of real-world fairness assessment. First, metric incompatibility: the three primary fairness metrics cannot all be simultaneously satisfied in most real-world settings where base rates differ across protected groups, a result established by Chouldechova (2017) and Kleinberg et al. (2016). Second, threshold indeterminacy: different regulatory regimes, protected characteristics, and system risk tiers require different threshold levels, and no universal threshold table is consistent with all applicable obligations. Third, conflict indeterminacy: when a system satisfies one metric but not another, no technical criterion determines which metric takes precedence. The resolution is inherently a value judgement that a governance framework must make explicit and accountable. This paper introduces the Fairness Threshold Governance Framework: three interlocking governance obligations that together produce a structured, auditable, and defensible resolution of the Fairness Threshold Problem. The Primary Metric Selection obligation requires the governing body to designate, prior to deployment, which fairness metric takes precedence for each system and each protected characteristic group, and to document the governance rationale for that selection. The Threshold Calibration obligation requires that thresholds be set by system risk tier and protected characteristic, with defined escalation obligations when thresholds are breached on any metric. The Conflict Resolution Protocol specifies the governance process for resolving cases in which primary metric compliance is achieved, but secondary metric breach occurs, establishing that this is a Board-level governance decision with defined documentation, escalation, and disclosure implications. The paper analyses five major governance frameworks and regulatory regimes, demonstrating that none provides a structured resolution to metric conflict, nor does any impose a governance obligation regarding the selection of primary metrics.
Authors
- M Maruf Hossain
Institutions
- RED Consulting (Norway) (NO)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-06-17
- DOI
- https://doi.org/10.5281/zenodo.20728390
- Primary Topic
- Ethics and Social Impacts of AI
- Type
- preprint