FedDPP: A Decentralized Peer-to-Peer Federated Learning Framework

The promise of maintaining a high classification accuracy while still achieving low communication and per-node computation cost, remains a central area of concern in federated learning protocols. We propose FedDPP, a Federated Decentralized Peer-to-Peer Mentor-Student learning framework, that is based on peer-to-peer communication among the neighbouring nodes. In this protocol, a node first queries the softmax values of an unlabeled data point for a random class label from its neighbours. Based on these values it identifies a superior mentor and adopts its parameters. The parameters are then retrained with local data. This leads to successful knowledge distillation across the network over the epochs with a small communication cost and without needing a central server to manage aggregation and communication. Experimental results show comparable accuracy across nodes while significantly reducing the overall training time by up to 2.5x and data communicated over the network by up to 3.5x in comparison to established federated learning setups.