BiasNet: A Framework for Unsupervised Discovery and Quantification of Bias in Datasets

Abstract - This paper introduces BiasNet, a comprehensive, end-to-end data analysis tool designed to uncover hidden biases and inequities in datasets without requiring pre-existing labels. BiasNet serves as an exploratory engine, leveraging a suite of unsupervised machine learning algorithms to segment a population into naturally occurring groups. By analyzing the demographic composition and characteristics of these discovered clusters, the tool quantifies potential disparities across sensitive attributes. The methodology encompasses versatile data ingestion, advanced preprocessing for structured and text data, a comprehensive suite of clustering algorithms, and the calculation of unsupervised fairness metrics. The entire workflow is encapsulated in an interactive web interface, culminating in a detailed, AI-generated PDF report with rich visualizations, making data auditing accessible to data scientists, analysts, and decision-makers. Key Words: unsupervised learning, bias detection, algorithmic fairness, data auditing, clustering, explainable AI, responsible AI