Exploring Fairness Debt Through Evidence from Studies on Algorithmic Discrimination

Context. The increasing deployment of artificial intelligence in decision-making processes has highlighted biases embedded in algorithms, leading to discriminatory outcomes that impact society. This phenomenon has led to the proposal of fairness debt, a type of software debt that emerges when biases are not addressed early in development and accumulate over time, creating costly, complex issues that can perpetuate societal inequities. Goal. This study aimed to explore the concept of fairness debt within algorithm-driven decision-making systems, identifying how deferred fairness considerations lead to accumulated biases and affect society. Method. We conducted a mapping study, analyzing 86 papers on algorithmic discrimination to classify root causes, effects, and instances of discrimination associated with fairness debt. Findings. Our findings indicate that several root causes, such as training bias, historical bias, and design bias, contribute to the accumulation of fairness debt, with manifestations in issues like sexism, racism, ableism, and ageism within systems. These accumulated biases can exacerbate social inequalities, limit algorithmic reliability, and perpetuate stereotypes, demonstrating fairness debt’s extensive societal impact. Discussions. Our study demonstrates that fairness debt extends beyond technical issues, encompassing broader societal consequences that demand proactive mitigation strategies, emphasizing the risks associated with deferring fairness considerations and highlighting the importance of fairness as a foundational principle in modern software systems, in particular AI systems. Conclusion. We explore algorithm discrimination through the lens of fairness debt, showcasing e that social concerns of algorithms can be interpreted as main component of software engineering practice, considering the socio-technical factors embedded in it and helping researchers and developers exploring discrimination in their systems and practices.