LSTM-Enhanced Predictive Display for Wireless Underwater Teleoperation Under Time Delays

Acoustic communication allows underwater Remotely Operated Vehicles (ROVs) to be controlled wirelessly, improving their maneuverability by removing the need for a physical tether connection to the surface. However, acoustic communication suffers from severe delays, degrading the operator's performance due to the lag between the given command and the vehicle's corresponding movement. In this paper, a novel delay mitigation approach is introduced, utilizing a Predictive Display (PD) enhanced by a Long Short-Term Memory (LSTM) neural network. The PD predicts the robot's delayed movements using dynamic modeling and LSTM-based time series forecasting. It then displays a virtual robot in real-time based on the operator's commands, which the real robot follows after a delay. Experiments are conducted on an acoustically-teleoperated, bioinspired robotic fish in our facility's pool with time delays reaching 2.5 seconds. A Visual Positioning System (VPS) is utilized to observe the robot's poses as ground truth. The LSTM network is trained to minimize the error between the robot's pose from its dynamic model and the ground truth. The corrected pose is displayed to the operator on the PD, with a haptic cueing device alerting the operator to potential collisions with obstacles. The results show average reductions of 82.3% and 65.24% in pose and angular errors, respectively, when utilizing the LSTM network compared to solely relying on the dynamic model. The code and dataset are available at: https://github.com/MahmoudElMezain/LSTM_PD