Estimating the Training Time in Single- and Multi-Hop Split Federated Learning

Split Federated Learning (SFL) is an upcoming and promising approach that balances the two main goals of distributed training, i.e., (i) the data remains at the data owners, and (ii) even devices with resource limitations can participate in the training. This is achieved by splitting the model into multiple parts and offloading them to designated compute nodes. Recent findings show that the number of compute nodes (hops) plays a significant role in the training delay. However, determining the ideal number of hops is not an easy task. Therefore, in this work, we propose a mathematical model that estimates the training delay of single- and multi-hop SFL. This tool not only helps in searching the optimal number of hops before the real deployment happens but also can be used as a lightweight evaluation tool in future research works in SFL. Our numerical evaluations show that the model can make correct estimations with an error smaller than 3.86%. Finally, we have constructed a lightweight optimization problem that finds the optimal cut layers (split points) and model part assignment to minimize training delay.