Optimal Energy Mix Modeling Simulation Using Reinforcement Learning on Philippine Monthly Energy Utilization for 2020–2022

A traditional method of generating an optimal energy mix involves selecting from various power producers, each with different costs per kilowatt-hour (kWh). This energy is then transmitted and distributed among various end-user sectors. However, this study explores a novel approach by replacing the traditional involvement of transmission and distribution utilities with reinforcement learning (RL) techniques. The objective was to determine an optimal energy mix over a 36-month period in the Philippines, from 2020 to 2022. The study evaluated several RL methods, including Q-learning, Deep Q-Network (DQN), Double Q-learning, and Actor-Critic algorithms. Each algorithm was tested with different learning rates <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(\alpha)$</tex> and discount factors <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(\gamma)$</tex> to find the most effective combination. Among these, the Actor-Critic algorithm, with <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\alpha=0.10$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\gamma=0.90$</tex>, emerged as the most optimal. The simulations showed that using each algorithm, no energy demand in kWh was left unmet, demonstrating its effectiveness in managing and optimizing energy distribution. This approach not only automated the decision-making process but also enhanced efficiency by continuously learning and adapting to changes in energy production and consumption patterns.