Reinforcement learning approach for optimal DoS attack power allocation: a WDSQL method

This paper develops an optimal denial-of-service (DoS) attack strategy for cyber-physical systems (CPS) under power constraints and limited prior knowledge. Unlike prior works that assume linear systems with steady-state Kalman filter convergence, we consider a nonlinear system in which the attacker has partial knowledge of the physical dynamics required for covariance propagation, observes ACK signals, but lacks wireless-channel information and transition probabilities. The objective is to learn a power allocation policy that maximises estimation performance degradation over a finite horizon. We formulate this problem as a Markov decision process (MDP) and propose a model-free algorithm, weighted double speedy Q-learning (WDSQL). WDSQL employs a weighted double estimator to balance both overestimation and underestimation biases, and incorporates auxiliary memory functions to accelerate convergence. We establish almost-sure convergence under finite-state and infinite-visitation assumptions and the learning-rate, conditional-expectation, and conditional-variance conditions stated in the convergence analysis, and derive conditional stability and error-propagation bounds under an additional boundedness condition, with a computational complexity analysis. Simulations on a dynamic target tracking scenario show that WDSQL achieves the highest attack effectiveness among the compared RL methods and a lower empirical running time than DQN under the tested setting. A proactive defense experiment further highlights the practical value of understanding learning-based attack strategies.