Real-time Tracking in a Pull-based Status Update System with Unreliable Command Channel

We consider a pull-based status update system consisting of a finite-state Markov source, an energy-harvestingenabled transceiver, and a sink. Both data and command channels are error-prone. The sink is interested in real-time tracking of the source and sends commands for updates to the transceiver. However, due to noise in the command channel, the transceiver may experience false alarms or miss-detections. We study the problem of minimizing the long-term time average of a (generic) distortion subject to a constraint on the average number of commands. Due to the pull-based nature of the system, the controller (i.e., sink) has only partial observability of the source state and the available energy at the transceiver. Therefore, we model the problem as a constrained partially observable Markov decision process (C-POMDP), which is then cast as a constrained belief-MDP (C-belief-MDP) problem. The infinite belief-state space makes solving the C-belief-MDP difficult. Thus, we effectively truncate the belief-state space and formulate a finite-state Cbelief-MDP problem. We develop a stationary randomized policy (SRP) to solve the problem by employing Lagrangian relaxation, the relative value iteration algorithm (RVIA), and bisection search. Due to the curse of dimensionality in deriving the SRP, we also develop a low-complexity policy applying the drift-pluspenalty method, which transforms the C-belief-MDP problem into a sequence of per-slot optimization problems. Simulation results show the effectiveness of the proposed policies and their superiority compared to a baseline policy.