Efficient Client Selection for Asynchronous Federated Learning for Adaptive Bitrate Streaming

Recently, Deep Reinforcement Learning (DRL) has been applied to enhance the Quality of Experience (QoE) of Adaptive Bitrate Streaming (ABR) by adjusting the video quality level in real time based on instantaneous network conditions. To build a state-of-the-art DRL-based ABR (DRLABR) algorithm, it must learn from the clients’ actual network and video streaming behavior. However, collecting such data directly from clients introduces several challenges, including privacy concerns, high bandwidth consumption, and the straggler effect—where poor network conditions of certain clients delay the training process, as DRLABR’s performance is highly dependent on network interactions. To overcome these limitations, we propose a decentralized training approach for DRLABR using a Federated Learning (FL) framework. Instead of gathering raw data, clients train their local DRLABR models independently and send only model updates to the central server. To address the straggler issue, we propose to desynchronize the FL update rules, allowing clients to contribute their model updates at their own pace, regardless of varying network conditions. In addition, we design a DRL-based client selection mechanism to prevent oversampling of high-bandwidth clients, which could lead to model divergence, thereby ensuring balanced participation and improving the overall training efficiency. We validate our approach through a comprehensive simulation encompassing diverse video content and real-world network traces, simulating a wide range of streaming activities. Our results show that the proposed framework significantly outperforms conventional FedAvg and FedAsync methods, achieving the highest average QoE score of 2.02 and reducing the total training latency by 21.26%.