Finding the Optimal Policy to Provide Energy for an Off-Grid Telecommunication Operator

We analyze a networking system powered by solar panels, where the harvested energy is stored in a battery that can also be sold when fully charged. Then the networking operator faces dual objectives: maintaining the functionality of its infrastructure and selling (or supplying to other networks) the filled batteries. These two goals are contradictory as selling the battery's energy may result in operational disruptions (e.g., packet delays) during certain periods. To address these challenges, we have developed a Markovian Decision Process (MDP) model that integrates positive rewards for battery release as well as penalties for energy packet loss and battery depletion. From this modeling, we present the optimal policy that balances these conflicting objectives and maximizes an average reward function. We advocate that integrating the particular structure of the MDP will enhance efficiency and precision of the numerical analysis. We provide numerical comparisons from small-scale to large-scale models and present a detailed analysis of agent behavior under the optimal policy.