Learning Adaptive Control of a UUV Using a Bio-Inspired Experience Replay Mechanism

This paper provides an in-depth analysis of the present state of Deep Reinforcement Learning (DRL) applications in Unmanned Underwater Vehicles (UUVs). Addressing the persistent challenges related to data inefficiency and performance degradation in physical platforms, particularly when faced with unforeseen variations, the paper introduces the innovative Biologically-Inspired Experience Replay (BIER) method. This approach incorporates two distinct memory buffers to enhance learning efficiency. The paper assesses the generalization capabilities of BIER through training neural network controllers on diverse tasks, spanning from inverted pendulum stabilization to simulating half-cheetah running. Furthermore, BIER is integrated with the Soft Actor-Critic (SAC) method for UUV stabilization under unknown environmental dynamics. Evaluation in a ROS-based UUV simulator, incorporating increasingly complex scenarios, showcases BIER's superior performance over traditional Experience Replay (ER) methods, achieving optimal UUV control in half the time. This review contributes valuable insights into the challenges and advancements in applying DRL methods to UUVs, highlighting the BIER method's promising potential to improve adaptability and efficiency in UUV manoeuvring tasks, leading to more robust and agile underwater vehicle control systems for more robust and agile underwater vehicle control systems. .