An Analysis of Network Overhead in Distributed TinyML

This paper presents an implementation of a distributed TinyML system and an analysis of the network overhead in communication between devices. We use a pipelined approach, where each device computes a portion of the layers and sends intermediates to the next device. Our results show that the communication overhead strongly dominates, taking over five times longer than computation time. We present possible solutions to mitigate this overhead, including smart partitioning of models, optimizing device placement and signal strength, and alternate wireless protocols such as WiFi. These strategies aim to make the deployment of several TinyML devices working together a more practical scenario.