Bias och Rättvisa inom AI i Hälso- och Sjukvård : En Fallstudie om Medicinsk Bildklassificering med Multi-Modal Grafnätverk

The integration of artificial intelligence (AI) in healthcare has led to significant advancements in medical imaging and diagnosis. However, AI-based medical image diagnostic models often exhibit biases due to demographic data imbalances, raising concerns about fairness. Many prior methods struggle to outperform traditional statistical methods and fail to leverage the rich complementary information in multimodal medical data, which resembles human diagnostic reasoning. To bridge this gap, this thesis explores a novel fairness-aware multimodal GNN framework as a case study to investigates the potential of multimodal strategy in mitigating bias while maintaining diagnostic utility in medical image classification. This research proposes two models: Multi-Modal GNN and Multi-Modal Multi-Channel GNN. These models incorporate three key modules: Cross- Modal Feature Fusion, Adaptive Graph Construction, and GNN-based Classification. Furthermore, the Multi-Modal Multi-Channel GNN constructs adjacency matrices from diverse modality combinations, enhancing intra- and inter-modal feature learning through inter-channel fusion. In pilot experiments on citation networks (Citeseer and DBLP), the Multi-Channel GNN outperformed baselines, including GCN, GAT, and GCNII, by up to 6.31% and demonstrated strong scalability. In the main experiments, the multi-modal configuration consistently outperformed the uni-modal setup on GNN-based diagnostic utility metrics (e.g., AUC, TPR@80TNR) and fairness metrics (e.g., AUC Gap, EqOdd). This demonstrates that integrating multi- modal information not only enhances disease feature detection but also ensures more balanced performance across different protected groups. Moreover, taking minimax fairness, group fairness and utility into account, the proposed Multi-Modal Multi-Channel GNN achieved competitive rankings (4/15 in the sex experiment and 5/15 in the race experiment), with only two SOTA methods—DomainInd and ODR-consistently outperforming it. This study underscores the potential of multimodal learning in addressing fairness while preserving medical image diagnostic accuracy in AI healthcare, providing insights and inspiration for future research.