Remote State Estimation With Discounted Multi-Armed Bandits for Non-Stationary Channel Selection

This paper addresses the problem of optimal channel selection for remote state estimation in cyber-physical systems, where a sensor transmits measurements over multiple time-varying wireless channels. We model the packet arrival probability of each channel as a non-stationary Bernoulli process and adapt two discounted Multi-Armed Bandit (MAB) algorithms, Discounted Upper Confidence Bound (D-UCB) and Discounted Thompson Sampling (D-TS), to select channels with the highest expected packet arrival rates adaptively. The estimation error covariance is analyzed using Kalman filtering, and the cumulative estimation regret is defined as the excess trace of the estimation error covariance compared to an optimal policy. Theoretical analysis shows the algorithms achieve a regret that grows gradually over time, and numerical simulations validate their effectiveness under non-stationary conditions.