A reinforcement learning based decision support system for multi-stage rooftop solar panel investment in a renewable energy community

In this article, we study the problem of rooftop solar panel installation, within the context of a newly commissioned renewable energy community. In particular, we consider a multi-stage, multi-investor setting, wherein each investor has to decide on the number of rooftop solar panels to be installed along the planning horizon, subject to uncertainty in the membership status of the community end-users. We model this membership evolution as a Markov chain and formulate the investor’s decision-making as a stochastic optimal control problem, with coupling in the objective function due to non-linear proportional sharing mechanism. We then translate the problem as a partially observable Markov decision process and propose a reinforcement learning based solution, which utilizes policy gradient architecture alongside parameter sharing. The proposed algorithm, named as advantage actor–critic+parameter sharing (A2C+PS) acts as a decision support system, i.e., it utilizes the localized observation as input and recommends to the investor, the number of rooftop solar panels to be installed. We perform Monte Carlo simulations utilizing the model built using the real-world household dataset and compare A2C+PS with independent advantage actor–critic (I-A2C), multi-agent advantage actor–critic (MA-A2C) and a rule-based strategy. Speaking statistically, the A2C+PS based decision support system results in ≈ 15 % increase in the number of scenarios where the investment cost was recovered within the planning horizon, with ≈ 3 % reduction in the first payback period.