Algorithmic Bias Detection: A Focus on Skin Tone and Gender Fairness in AI Models

This study delves into the detection of algorithmic bias and fairness benchmarking in Artificial Intelligence (AI) and Machine Learning (ML) systems, particularly within Computer Vision applications, with an emphasis on how skin tone variations affect model performance. We analyze gender prediction among leading AI services such as Amazon Rekognition, Deep Face, Fair Face, and the BadgerCake gender-classifier by applying fairness metrics and conducting bias-focused studies across various skin tones. A novel aspect of our methodology involves mapping detected facial attributes and skin tones using a predefined scale, enabling a detailed performance analysis across gender segments. Image preprocessing techniques, including contrast enhancement, normalization, and speckle removal, are applied to improve image quality and enhance the reliability of skin tone detection. Using metrics such as equalized odds, calibration, demographic parity, and accuracy parity, our comparative analysis reveals significant disparities, including high accuracy for AWS Rekognition and pronounced biases in Fair Face and BadgerCake for darker skin tones. These findings highlight the need for data augmentation, algorithmic adjustments, and diverse training datasets to mitigate biases and refine algorithms. This research underscores the critical need for continuous evaluation and improvement of AI systems to achieve fair and accurate outcomes across diverse populations.