Distributed swarm learning for communication-efficient capsule endoscopy classification on solving Non-i.i.d data

The data private concern of centralized medical analysis models has gained more attention in recent years. With the improvement of legislation on sharing sensitive information, public medical datasets are no longer available, blocking the usage of Deep Learning in clinical diagnosis, especially in capsule endoscopy classification. Although the distributed learning model allows training locally without data sharing, some key challenges still restrain their real-world application. In this paper, we focus on the distributed training framework for the CE classification model and propose a multi-client distributed swarm learning framework, named Joint Distributed Swarm Learning (JDSL), to alleviate the impact of data heterogeneity and communication cost. To relieve the degradation of non-i.i.d. input on stochastic gradient descent, we introduce a distributed swarm learning structure to optimize the training process by swarm theory. To enhance communication in cooperative learning, we proposed a multi-client selection strategy to aggregate high-performance local models for the global model. To validate the proposed framework and explore how the non-i.i.d. dataset affects distributed learning, we utilize the Dirichlet process to continuously generate datasets with different distributions. Results on different scales of non-i.i.d. datasets show the remarkable performance of JDSL in tackling heterogeneous datasets with less communication round.