An Investigation into Open Source Fairness Tool Sustainability

As our society grows increasingly reliant on artificial intelligence, the need to mitigate risk and harm has become paramount. In response, researchers and practitioners have rallied to provide tools that help mitigate risks and harms by detecting and mitigating un-desired bias, commonly known as fairness tools. Many of these tools are made available free to the public for use and adaptation. The increase in available tooling is promising, however, little is known about the landscape of tools available beyond the ones commonly discussed (e.g., AI Fairness 360, Fairlearn). Since fairness is an ongoing necessity, the tools must be built for long-term sustainability. Using an existing set of fairness tools as a guide, we systematically searched for the current state of art in GitHub and identified 50 projects. We then analyzed various aspects of these tools' repositories to assess the community engagement (are developers using or engaging with them) and maintenance of these tools in practice. Our findings suggest diverse kinds of engagements with these tool repositories, indicating their embrace of open source. Furthermore, we found that some tools are more well-maintained than others, which may affect usage in practice. Our findings revealed that 53% of fairness projects become inactive within the first three years.