Fairness in Generative AI: A Framework for Evaluating Demographic Bias in Text-to-Image Models Through Occupational Data

AI-generated images can inherit and amplify biases from their training data, leading to skewed representations of gender and ethnicity. As Text-To-Image (TTI) modelsbecomemorewidelyadopted, itiscrucialtoidentifyandaddressthesebiases to ensure equitable and fair representations in digital media. This thesis investigates gender and ethnicity biases in TTI models by analysing their representation of different occupations. A set of popular occupations was selected based on their frequency in large text corpora. For each occupation, a large set of images was generated using Stable Diffusion [1] and FLUX [2] models and automatically classified through an available unbiased method to extract demo- graphic attributes such as gender and ethnicity. A comprehensive framework was developed to manage the entire process, from image generation to validation and classification. It computes bias metrics to assess the over and underrepresentation of demographic groups in generated data, ultimately enabling interactive visualisa- tion for deeper analysis. Furthermore, the framework facilitates comparisons with real-world statistics, enabling an assessment of the model’s bias with respect to the actual demographic distributions. Although the framework is designed to operate with real-world data, we showcase its reliability and effectiveness using synthetically generated data simulating “real-world” statistics. A series of experiments conducted using the proposed framework demonstrates that state-of-the-art TTI models exhibit a stereotyped and biased representation of the world, and allows for the automatic assessment of these aspects in additional models with respect to specific populations.