An Efficient Speculative Federated Tree Learning System With a Lightweight NN-Based Predictor

Federated tree-based models are popular in many real-world applications owing to their high accuracy and good interpretability. However, the classical synchronous method causes inefficient federated tree-based model training due to tree node dependencies. Inspired by speculative execution techniques in modern high-performance processors, this paper proposes FTSeir, a novel and efficient speculative federated learning system. Instead of simply waiting, FTSeir optimistically predicts the outcome of the prior tree node. By resolving tree node dependencies with a neural network-based split point predictor, the training tasks of child tree nodes can be executed speculatively in advance via separate threads. This speculation enables cross-layer concurrent training, thus significantly reducing the waiting time. Furthermore, we propose an eager verification mechanism to promptly identify mispredictions, thereby reducing wasted computing resources. On a misprediction, an incomplete rollback is triggered for quick recovery by reusing the output of the mis-speculative training, which reduces computational requirements. We implement FTSeir and evaluate its efficiency in a real-world federated learning setting with six public datasets. Evaluation results demonstrate that FTSeir achieves up to 3.45× and 3.60× speedup over the state-of-the-art gradient boosted decision trees and random forests implementations, respectively.