Learning Arrival Cost for Moving Horizon Estimation via Temporal Difference

For nonlinear state estimation problems, Moving Horizon Estimation (MHE) is a competitive alternative to various recursive Bayesian estimators. The MHE method considers observation information in a sliding window of fixed time length and approximates the past observation information as an arrival cost term. The goodness of the arrival cost approximation affects the estimation performance and the computational burden of the method. In this paper, we propose to approximate the arrival cost via temporal difference learning, aiming to improve the estimation performance of MHE while not imposing heavy computational burden associated of the arrival cost term. We establish the Bellman equation of the exact arrival cost terms, and derive that the update process of quadratic arrival cost can be seen as a nonlinear dynamic system, which can be represented by Recurrent Nerual Network (RNN). Simulation results show that the proposed method, compared with computing the arrival cost term using Extended Kalman Filter (EKF), offers better estimation performance and similar program execution time when the same window length is selected.