Advances in the application of model-free reinforcement learning in protecting transactive energy systems against cyberthreats: A review

• synopsis of core concepts on the suitability of the RL technique in solving identified cyber threat challenges in TES. • Objective analysis of core concepts on the suitability of the model-free RL approach in cyberthreats mitigation in TES. • Model-free RL in energy systems, with an emphasis on the Tasks, Algorithms, and Reported Novelties. • Coordinated cyber-attacks and potential impact on energy market operations in TES. The security and resilience of energy systems are essential in the reliable operation of emerging energy solutions such as the Transactive Energy System (TES). Integrating Information and Communication Technology components enhances the system reliability of the energy system. However, this integration introduces complexity due to increased digitization, which makes energy systems more vulnerable to cyber threats. The interconnectedness and interdependence of physical and digital infrastructures have been linked to the potential impact of cyberattacks on energy supply, making them susceptible to adversaries' attempts to disrupt services, access sensitive data, or seek undue financial gains. Therefore, this paper presents a review of current trends by examining model-free Reinforcement Learning (RL) methods in energy systems to identify the challenges that impede its application for detecting and mitigating cyber-attacks in a decentralized TES. It also provides a synopsis of core concepts regarding the suitability of RL techniques for addressing these identified challenges. The article further discusses innovative approaches to existing RL-based algorithms and their corresponding novelties. Findings reveal that RL is widely used in Energy Management Systems, but its application in mitigating cyberattacks in TES remains limited. Moreover, this work highlights that 51 % attacks, an emerging cyberattack, have not been extensively explored by researchers to evaluate their impact on blockchain-based TES and the potential role of RL in tackling this issue. Ultimately, selecting critical parameters and optimizing the neural networks in RL systems requires further attention.