Towards Proactive Fairness Monitoring of ML Development Pipelines

Fairness in machine learning (ML) algorithms is a multifaceted issue that necessitates input from technical, social, and legal perspectives. Ensuring effective bias mitigation requires input from multiple stakeholders throughout the development process. However, in practice, development teams, particularly those in small and medium-sized enterprises (SMEs), often lack the resources and expertise to thoroughly analyse and address fairness concerns within their product development workflows. To address this gap, it is crucial to create practical guides or blueprints for product development pipelines. These resources should enable each stakeholder to understand the issues of fairness, and where challenges can arise, as well as provide measurable techniques to address these issues. In this short paper, we explore the viability of a proactive and continuous fairness monitoring flow to seamlessly integrate bias identification and mitigation strategies in an existing product development pipeline. Further, we present open questions related to the implementation and use of this tool, inviting further discussion and research. We particularly focus on curating a set of best practices to establish an effective fairness monitoring flow, aimed to facilitate alignment in the bias mitigation strategies across different teams within the development process.