Enhancing Federated Learning With Differentially Private Continuous Data Release via <i>k</i> -Ary Trees

Applying differential privacy (DP) to federated learning (FL) effectively safeguards participants’ training data against privacy threats, yet the stringent availability requirements of FL present a significant challenge in optimizing accuracy while ensuring privacy. Integrating differential privacy continuous data release (DPCR) into private FL mitigates the errors accumulating on intermediate parameter models securely, thereby enhancing accuracy, with performance gains driven by developing advanced DPCR models. In this context, we propose a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i>-ary Tree-based DPCR (kTCR) model to provide deeper and more flexible error optimization, thereby promoting a robust accuracy enhancement for private FL. Our kTCR model introduces Variance Optimal Estimation (VOE) and privacy budget allocation (PBA) methods to optimize accuracy, posing significant efficiency challenges simultaneously. With rigorous mathematical analysis, we reduce the computational complexity of VOE from <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i>(<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</i><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup>) to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i>(lg <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</i>), then introduce a meta-factor method that transforms the challenging PBA issues into a convex optimization problem with significantly reduced variables (e.g., a 3-ary tree with 5.23 × 10<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">9</sup> nodes requiring only 61 variables), thus yielding high efficiencies. Our experiments on classical datasets demonstrate that our kTCR model with appropriate <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> outperforms the state-of-the-art ABCRG by 0.7% ~ 2.0% in accuracy and traditional private FL by 5.5% ~ 18.7%. Further experiments demonstrate that adjusting the arity <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> effectively reduces the Pre-aggregation Error, leading to a further 2.09% accuracy gain. Our achievements both improve the accuracy of private FL and provide new insights into building high-availability FL with DP.