Overmind: Fast and Scalable Decentralized Learning Simulation for Mobile Environments

Recent progress in decentralized machine learning provides a foundation for a future in which mobile devices collaboratively learn from the vast amount of crowdsourced data they produce without necessarily relying on a centralized server for coordination. However, testing novel decentralized learning algorithms requires many iterations that are very costly in terms of time and budget. We present Overmind, a fast simulation framework for testing the performance of decentralized learning algorithms in dynamic networks. The design of Overmind allows users to easily scale their scenarios in terms of both simulated nodes and the number of physical machines on which the simulation runs. Overmind supports an operational mode that expedites the simulation by approximating the interaction time between two nodes, which allows skipping events that are unlikely to happen (e.g, two nodes exchanging big model parameters on a connection that is maintained for a very short time period). Users can also configure Overmind with a dataset of connection traces of simulated nodes, which is especially helpful to test algorithms given realistic dynamic networks in which devices join and leave unpredictably. To leverage Overmind, users just need to implement the generic behavior that is invoked on participating nodes when they connect to or disconnect from other nodes. The implemented behaviors are invoked in Overmind according to a provided encounter trace. This node-centric programming model that Overmind provides, where users define node behavior based on an individual node's perspective, is an intuitive design for decentralized settings. In this paper, we discuss the design of Overmind and demonstrate that it enables fast, scalable, and easily configurable decentralized learning simulations.