Markov Decision Processes for Scheduling Lunar PNT Services

The renewed focus on lunar exploration, highlighted by over a hundred planned missions in the coming decade, calls for developing a dedicated lunar Positioning, Navigation, and Timing (PNT) and communication infrastructure. This need led to the conception of LunaNet, a collaborative network that facilitates interoperability among commercial and governmental entities through standard protocols and interfaces. Given the expected use of small satellite constellations with limited resources for initial lunar PNT infrastructure, the efficient scheduling of tasks and resources becomes paramount for ensuring reliable and timely services. Furthermore, the increased autonomy of future missions necessitates the development of User-Initiated Services (UIS) to allow assets to autonomously access network services, highlighting the dynamic nature of scheduling needs. This work introduces a novel scheduling framework for optimizing lunar PNT and communication services within the context of LunaNet. Relying on a Semi-Markov Decision Process (SMDP) formulation to incorporate multiple satellites while considering the dynamics of user requests and operational constraints, the proposed approach aims to optimize satellite task scheduling to ensure efficient resource use and service delivery. This framework considers various operational constraints, including onboard power and data limits, and is designed to adapt schedules dynamically to user requests. The proposed method is benchmarked against traditional approaches, such as rule-based and Integer Linear Programming (ILP) methods, to demonstrate its effectiveness in optimizing service delivery and resource utilization. The main contribution of this work is a flexible, efficient solution for scheduling lunar PNT services, accommodating the dynamic nature of lunar missions, and enhancing the autonomy of future lunar exploration efforts.