A Debiasing Framework For Attribute Binding In Diffusion-Based Text-To-Image Generation

Despite the impressive generative capabilities of text-to-image (T2I) models, accurate binding of objects and attributes specified in input prompts remains a significant challenge. Existing approaches often fail to address inherent biases in text embeddings, where objects tend to associate with their frequent attributes in the training data. This work identifies these biases and introduces a novel debiasing framework centered on alternating prompt vector binding, dynamically improving semantic alignment during image generation. First, we quantify bias strength in object-attribute pairs using large language models (LLMs), revealing problematic associations. Next, biased object terms are replaced with broader concepts during diffusion sampling to reduce bias. Finally, object embeddings are refined by pulling target attributes while suppressing irrelevant ones, avoiding attribute confusion. Extensive experiments on public datasets demonstrate the framework’s effectiveness in addressing attribute binding challenges through bias mitigation.