A Deep Reinforcement Learning Based Dynamic False Data Injection Detection in Load Prediction

The integration of smart meters into modern energy distribution systems has improved grid efficiency and consumer energy management. However, it also introduces new cybersecurity vulnerabilities, particularly False Data Injection (FDI) attacks that can manipulate energy consumption data. Traditional anomaly detection approaches often rely on supervised learning and require extensive labeled attack data, which may not generalize well to evolving cyber-threats. To address these limitations, this paper formulates the FDI detection problem as a partially observable Markov decision process (POMDP) and leverages the proximal policy optimization (PPO) reinforcement learning (RL) algorithm to solve the problem. A belief estimation model is developed using a multi-head self-attention architecture to represent the agent's uncertainty about whether the data point is attacked. The belief model, as a supervised learning model, is also used for comparing the results which proves the superior performance of the PPO-RL agent in detecting sequential and random scaling attacks at varying intensities. The PPO-RL agent also gains a higher reward than an RL agent trained based on the Advantage Actor-Critic (A2C) algorithm.