Sequence-Aware Online Container Scheduling With Reinforcement Learning in Parked Vehicle Edge Computing

Intelligent vehicles, often parked for long periods, are ideally suited to serve as computational nodes to expand the Mobile Edge Computing (MEC) infrastructure, with containerization significantly enhancing the system's load balancing, self-healing, resource isolation, and security. However, fluctuations in task demand and frequent container image downloads during peak hours create high loads on containerized nodes, as multiple mobile devices offload tasks simultaneously, leading to significant processing delays. Many existing studies make the simplified assumption of predefined patterns of task arrivals, which overlooks this issue and makes suboptimal decisions. In this paper, we consider a Parked Vehicles (PVs)-extended MEC scenario, where multiple devices request services on PVs functioning as edge servers, all controlled by a central base station. Task arrivals follow observed patterns based on long-term trends, such as peak and off-peak periods, resembling realistic arrival patterns rather than predefined ones. To optimize task offloading by identifying these patterns, we propose the Sequence-Aware Task Scheduling (SATS) algorithm, which is a policy gradient-based deep reinforcement learning approach that integrates Transformer and LSTM architectures to capture patterns in time-series task arrivals and relationships between nodes in a collaborative and containerized environment, thereby enhancing the efficiency of online task scheduling. The primary objective of SATS is to optimize the task offloading policy and minimize delay and energy consumption for all devices and PVs. Extensive numerical comparisons against baselines demonstrate the effectiveness and advantages of our algorithm.