Fv‐SFL: A Contrastive Learning‐Based Feature Sharing Method for Reducing the Effect of Label Skewed Data Heterogeneity in Federated Medical Imaging

ABSTRACT Deep learning plays a crucial role in medical image analysis. Traditionally, it involves the collection of patient images at a central location. For this reason, centralized approaches have encountered technical challenges such as data security vulnerabilities, data transfer bottlenecks, limited data diversity, and government regulatory hurdles like HIPAA and GDPR. Federated Learning presents an alternative approach by allowing model training without sharing patient data from client hospitals. However, it faces challenges such as label‐skewed data heterogeneity due to variations in population characteristics, biases, and disease prevalence among hospitals, which leads to performance drift during model training. We propose a framework called Feature vector sharing‐based Federated Learning (Fv‐SFL) to address this issue by combining a novel contrastive learning‐based feature‐sharing method and distribution‐discrepancy‐based aggregation. This introduces a local learning approach incorporating class‐wise feature vectors for federated learning. These vectors, defined as the average vectors of representations within distinct classes, allow for the utilization of clients' knowledge to refine local training. In addition to adjusting server aggregation, we integrate a distribution discrepancy method to calculate the weight for each client for server aggregation. We evaluate the effectiveness of our method for both multiclass and binary classification tasks by conducting experiments on two distinct datasets. Firstly, assess the method's performance on a multiclass classification task using the Ham10000 dataset. Secondly, evaluate its efficacy on a binary classification task using the COVID‐QU‐Ex dataset. Across various methods, Fv‐SFL consistently outperforms other federated learning methods, indicating its superior performance compared to alternative approaches. This framework effectively mitigates performance drift issues during model training caused by label‐skewed data heterogeneity by utilizing feature vector sharing‐based contrastive learning methods and discrepancy‐based global aggregation. Additionally, Fv‐SFL outperforms traditional FL methods by optimizing resource utilization with reasonable communication costs.