Planning Versus Learning: Fair Space-Time Scheduling for Unwired Networks

Space-time scheduling for multi-user networks under fairness considerations is investigated. Scheduling is formulated as a sequential decision-making problem under the Markov Decision Processes (MDP) framework. Although the initial focus of the work is underwater acoustic networks, the proposed strategies are also validated for terrestrial radio frequency networks. If environment exploration is expensive, planning is more efficient than online learning. A challenge of the proportional fairness is that the additive structure between current and future rewards does not hold. An approximate reward function that is additive is proposed, enabling dynamic programming. Computational complexity is addressed through sample-based approximations. Error accumulation and error bounds are analyzed to show that error decays with time. As mobility induces model-shifts, a novel re-planning scheme is proposed to optimize the timings of policy updates. Numerical results show that the proposed scheme significantly improves network capacity while maintaining a high level of fairness. Furthermore, the proposed approach yields average capacity and fairness gains as high as 37% and 27%, respectively, compared to current approaches.