Imitation learning from observation for ROV path tracking

Abstract Remotely operated vehicles (ROVs) are playing indispensable roles in the ongoing exploration and utilization of ocean resources as they offer flexibility and efficiency. Deep reinforcement learning (DRL) algorithms have been widely used to enhance ROV autonomy, reduce operator workload, and minimize human errors in operations. However, traditional DRL methods rely on a well-crafted reward function specific to the task, which is often challenging to design precisely. Learning from demonstration offers an alternative way, as it enables agents to imitate expert trajectories and refine their policies without relying on reward functions. However, although most existing studies assume that detailed action or control information is available from expert demonstrations, such data are typically hard to obtain in practice. To overcome this limitation, we propose and implement an imitation learning from the observation method for ROV path tracking. In our approach, policy learning is derived solely from observed expert trajectories without the need for explicit action data. We evaluated our method on both straight-line and sinusoidal tracking tasks, and compared the results to those of proximal policy optimization (PPO), a traditional DRL algorithm, using predefined rewards. The experimental results demonstrate that our approach achieves a performance comparable to that of PPO, while offering a faster learning rate and enhanced adaptability to different tasks.