Lyapunov-Guided Deep Reinforcement Learning for Computation Offloading in Ocean Internet of Things

The Ocean Internet of Things consists of heteroge-neous network nodes, creating complex challenges for optimizing computation offloading within ocean mobile edge computing. To address these challenges, a new offloading framework is introduced that leverages attention-based deep reinforcement learning combined with Lyapunov optimization. This framework effectively addresses the coupling issues in multi-frame decision- making. The approach begins by transforming the original multi-stage, mixed-integer nonlinear programming problem into a deterministic form for each time frame using Lyapunov opti-mization. Deep reinforcement learning is then applied to manage long-term constraints. Furthermore, to improve environmental perception and action control, a Convolutional Neural Network with an enhanced parameter-sharing Deep Neural Network architecture is used to approximate policy and value functions. Additionally, a squeeze-and-excitation attention mechanism is integrated to strengthen feature extraction capabilities. Extensive simulations validate the superiority of this framework. Comparative results show that, in contrast to a framework only incorporating a CNN with a squeeze-and-excitation mechanism, this enhanced framework achieves roughly a 17 % reduction in training loss and a 5 % decrease in energy consumption.