Minimizing the Age of Information in Status Update Systems With Multiple Sources of Uncertainty

Status Update System (SUS) are monitoring applications in the Internet of Things (IoT)They are formed by a sender that monitors a remote process and sends status updates to a receiver over a wireless data channel. The goal of the sender is to find a monitoring and transmission strategy that keeps the information at the receiver fresh, i.e., that minimizes the Age of Information (AoI) at the receiver. To be able to monitor and transmit at the optimal points in time, the sender needs to accurately track the quality of the data channel and the AoI at the receiver. The quality of the data channel is a source of uncertainty, as it is unknown to the sender. In fact, there is no possibility to be absolutely certain about the quality of the data channel at any time. The AoI at the receiver is only known at the transmitter when acknowledge (ACK) or negative acknowledge (NACK) feedback signals from the receiver are successfully decoded. However, in real applications, the feedback channel is a second source of uncertainty since it is prone to errors, thus the transmission of ACK/NACK messages might fail. Additionally, the random energy harvesting process is a third source of uncertainty. This means, the monitoring and transmission decisions have to be made amidst these multiple sources of uncertainty. To overcome this challenge, we introduce the so-called belief distribution and propose a novel joint monitoring and transmission strategy at the sender based on reinforcement learning. Our new approach, termed Continual Belief Learning, exploits the belief distribution to minimize the AoI at the receiver. Through extensive numerical simulations, we show that our proposed approach yields a significantly lower average AoI compared to state-of-the-art transmission strategies for AoI minimization in SUS.