Reles-OTA: A Reinforcement-Learning Enhanced Scalable Over-the-Air Update Approach for CAVs

Scalable over-the-air (OTA) updates for connected and autonomous vehicles (CAVs) present a significant challenge for many original vehicle equipment manufacturers (OEMs) due to the lack of scalability in their current software update solutions. We have proposed a novel reinforcement learning (RL) approach to tackle this issue. The problem-solving capability of deep reinforcement learning (RL) has been advanced in many fields, yet its potential for tackling over-the-air (OTA) update problems remains under-explored. As one of the backbones for enabling fast software iterations in various autonomous systems and robotic platforms, the OTA update problem remains challenging. This work primarily focuses on two key aspects of the OTA challenge: data payload size and memory overhead. By formulating the OTA problem and reducing it to a sequential codec construction problem and a Markov-Decision Process environment, we can apply deep reinforcement learning algorithms and generate approximately optimal OTA payload with small payload size and memory usage. Running experiments on real-world datasets, our proposed method performs better than open-source methods, effectively showcasing the potential of RL-augmented OTA update algorithms to reduce payload size and minimize runtime memory overhead. Finally, this work lays a robust foundation for applying deep RL in automotive OTA update systems, with potential implications for broader autonomous systems and robotic platforms.