Putting People First: A Multimodal Bi‐Stream Structure‐based Model for Bias Mitigation

ABSTRACT Research into multimodal model biases has largely concentrated on gender and racial issues, neglecting key areas such as biases related to religion, nationality, sexual orientation, and disabilities, as well as the subtleties of question intent and common biases. To address these gaps, we developed the Multimodal Bi‐Stream Structure‐based Model (MBSS) for Bias Mitigation, an innovative approach that focuses on these underrepresented groups by focusing on religion, nationality, sexual orientation, and disability but also to identify common biases and question intents. The MBSS employs a dual stream bias mitigation system, where the standard stream addresses common biases and the mitigation stream targets specific biases, allowing for nuanced and effective bias reduction by making decisions based on the difference in predicted probabilities between these two streams. Empirical results demonstrate MBSS's efficacy, especially in reducing religious bias by up to 71%, highlighting its potential to foster a more inclusive digital environment and enhance the representation of diverse groups.