Pareto DQL-MultiMDP Sub-Controllers for Load Balancing in Large and Dynamic WiFi Networks

This paper extends a framework for load balancing in large Wi-Fi networks we previously proposed. The framework partitions the network into clusters and assigns a sub-controller to each cluster. The sub-controllers employ a Deep Q-Learning-based algorithm to balance the load on the access points in the cluster. The sub-controllers collaborate by exchanging their training updates through a database. However, in highly dynamic Wi-Fi networks, the frequency of these exchanges may cause control plane overhead. In this paper, the training updates are uploaded only when their “quality” meets a condition dependent on a threshold. High threshold values reduce overhead but compromise local learning performance, and vice versa. Therefore, finding the optimal threshold values is formulated as a multi-objective optimization problem. Two metrics are designed to quantify the overhead and instability. These metrics are stochastic, scenario-dependent, and time-consuming to estimate. Randomly generated scenarios of large and dynamic 802.11ax networks are simulated to collect realizations of the metrics. The objectives are obtained by smoothing the realizations with Gaussian kernel regression. The problem is then solved with a genetic algorithm to estimate the Pareto-optimal threshold values.