Enhancing Maritime Remote Control via Edge-Cloud Computing: A Bifurcated Vehicle Control Approach

This paper introduces a bifurcated vehicle control architecture integrating edge and cloud computing. It aims to enhance maritime remote control operations, addressing the slow-changing dynamics due to climate and water flow, and limited network coverage. Our approach utilizes Deep Reinforcement Learning (DRL) to adapt to non-ideal, model-free scenarios. The edge server provides immediate control commands based on resident models, while the cloud server employs life-long learning to adapt to changes, continuously improving model accuracy. Specifically, we adopt the Soft Actor-Critic (SAC) algorithm under a discrete-time system, where the entropy in its loss function encourages the agent to explore the environment and exhibits greater stability when the controlled process dynamics change. Building on SAC, we propose the concept of Version of Model (VoM) and develop a weighted sampling strategy that adjusts the sampling probability for lifelong learning on the cloud server. We also introduce a dual-window updating strategy that surpasses standard updates by using a sliding and freeze window mechanism, minimizing unnecessary adjustments. The bifurcated control method, in conjunction with the dual-window update strategy, ensures timely and accurate process control, with effective correction for deviations. Finally, the efficacy of our approach is validated through simulations using a pendulum model and a Remotely Operated Vehicle (ROV).