Prompting Medical Vision-Language Models to Mitigate Diagnosis Bias by Generating Realistic Dermoscopic Images

Artificial Intelligence (AI), specifically deep learning has made significant advancements in skin disease diagnoses. However, a major concern with deep learning-based models is the biased performance across subgroups, particularly regarding sensitive attributes like skin color. Toward mitigating such diagnosis biases, we propose a novel generative AI-based framework, namely Dermatology Diffusion Transformer (DermDiT). DermDiT leverages text prompts generated via large vision-language models and multimodal text-image learning to generate new dermoscopic images. Through an effective prompting, DermDiT can generate realistic synthetic images leading to improved representation of underrepresented groups in highly imbalanced datasets for clinical diagnoses. Extensive experimentation showcases that our innovative prompting in DermDiT provides more insightful representations to generate high-quality and useful images. Our code is available at https://github.com/Munia03/DermDiT.