Modelbaserad reinforcement learning för kort- och långtidsoptimering av carriers

As the energy demand increases globally along with the demand for cellular traffic, the importance of energy efficient communication networks is ever important. Radio access networks (RANs) only utilize a fraction of their capacity while consuming the majority of the energy in cellular communication. Therefore, sleep mode (SM) optimization has the potential to save a lot of energy. In this thesis, we develop a framework for jointly optimizing short- and long-term sleeping of carriers in a RAN with key performance indicator (KPI)-constraints by iteratively optimizing short-term sleeping locally for each carrier and then the global user association for the network together with the long term sleeping. We develop a local semi-Markov queuing based traffic model over which short-term sleeping can be optimized using entropy-based reinforcement learning (RL) algorithm soft actor-critic (SAC). Additionally we develop an algorithm for adjusting traffic parameters for the local model based on the user-association and simulated traffic data from a 3rd Generation Partnership Project (3GPP)-compliant simulator. The short-term sleeping policy was able to outperform baseline metrics without violating the KPI-constraint for the evaluated case and the estimated model parameters were able to capture the overall traffic behavior from the simulated traffic data.