Federation Opacity and the Promise of Federated Learning in Healthcare

Federated learning (FL) is a machine learning (ML) approach that allows multiple devices or institutions to collaboratively train an ML model without sharing their local data with a third-party. It has recently received significant attention as a promising way to overcome longstanding ethical obstacles to training medical ML models with patient health data. This paper examines the promise of FL in healthcare from an ethical perspective. It argues that medical FL generates a new variety of opacity - <i>federation opacity</i>, wherein stakeholders cannot access, analyze, or curate the data on which a model has been trained - which (a) presents distinctive ethical challenges concerning <i>institutional</i> fairness and accountability in medical ML; and (b) makes FL models especially vulnerable to data poisoning attacks. It then identifies several key claims about the expected benefits of FL in healthcare and argues that they may be either exaggerated, misleading, or incomplete - often due to the problem of federation opacity.