Formulation and Simulation of Optimal Energy Mix Model Using Reinforcement Learning Methods With a Test Case on Philippine Monthly Energy Utilization Data for 2020-2022

Traditional methods for generating an optimal energy mix in the Philippines involve selecting from various power producers, each with different costs per kilowatt-hour (kWh). This energy is then transmitted and distributed among various end-user sectors. This study explores a novel approach by leveraging reinforcement learning (RL) techniques to optimize the energy mix. The main objective is to establish a model of energy mix that could be optimized when appropriate Reinforcement Learning methods are employed. The subsequent specific objective for the application is to determine an optimal energy mix over a 36-month period (2020-2022) in the Philippines using the established model. The RL techniques evaluated include Q-learning, Deep Q-Network (DQN), Double Q-learning, and Actor-Critic algorithms, each tested with different learning rates (Alpha) and discount factors (gama). The data used encompasses monthly energy generated (in GWh) from various sources (coal, oil-based, natural gas, and renewables), monthly energy consumption (in GWh) by sector (residential, commercial, and industrial), and the average monthly price (in Philippine Peso, PhP) across the country. The Actor-Critic algorithm with alpha = 0.10 and gama = 0.90 emerged as the most effective in optimizing the energy mix. The simulations showed that all RL algorithms met the energy demand without any shortages, demonstrating their potential in enhancing energy management by continuously adapting to changes in energy production and consumption patterns. This approach not only automates decision-making processes but also improves efficiency by utilizing historical data and addressing the fluctuating nature of energy markets.