Reward Systems for Trustworthy Medical Federated Learning

Federated learning (FL) is a popular technique to train machine learning (ML) models for healthcare where institutions collaborate by sharing ML model gradients. An undesired phenomenon in ML models is bias, which may cause unfairness against specific subgroups. In FL research, it remains unclear to which extent bias occurs, and how to best incentivize institutions contributing toward trustworthy FL models. Besides bias, trustworthiness aspects include high predictive performance. Existing reward systems, incentivizing predictive performance only, can transfer model bias against patients to an institutional level. Therefore, we evaluate how to measure the contributions of institutions toward either predictive performance or bias in FL and design corresponding reward systems, before we propose a combined reward system that incentivizes both. We evaluate our work using multiple chest X-ray datasets focusing on sex- and age-related patient subgroups. Our results show that we can successfully measure contributions toward bias, and an integrated reward system successfully incentivizes contributions toward a well-performing model with low bias. Thereby, institutions with data predominantly from one subgroup introduce a favorable bias for this subgroup. In a label flip experiment, we furthermore show that reward systems can incentivize institutions to ensure a high dataset quality.