Deadline-aware service scheduling via multi-head MARL in device-edge-cloud environments

• Introduce a cooperative multi-agent reinforcement learning (MARL) framework that jointly optimizes service scheduling across device, edge, and cloud layers, addressing dynamic and heterogeneous Mobile Edge Computing (MEC) environments. • Design a service-type-specialized multi-head Q-network architecture that efficiently manages heterogeneous IoT workflows with varying task numbers, deadlines, and resource demands. • Propose a Double DQN variant (MH-DDQN) that improves convergence speed, training stability, and deadline compliance compared to classical DQN. • Demonstrate through simulations that MH-DDQN achieves up to 85% deadline compliance under heavy load, outperforming heuristic baselines by over 25%, matching or slightly exceeding an adaptive SA benchmark, and simultaneously maintaining lower latency and adaptive task distribution. Mobile Edge Computing (MEC) enables latency-sensitive Internet of Things (IoT) applications by offloading computation to nearby edge servers. However, most existing intelligent scheduling approaches either neglect the full three-tier device–edge–cloud architecture or fail to account for heterogeneous IoT services composed of multiple dependent tasks with diverse deadlines and resource demands. These gaps hinder adaptability under dynamic network conditions. We propose a multi-agent reinforcement learning (MARL) framework in which cellular IoT devices and the edge server act as cooperative agents to optimize task offloading and scheduling. Each agent employs a multi-head deep Q-network (MH-DQN)—with one head per service type—to efficiently manage heterogeneous service workflows. We further implement a multi-head Double DQN (MH-DDQN) variant to improve stability and convergence. In addition, we benchmark our approach against four heuristic baselines and an adaptive simulated annealing (SA) scheduler. Simulation results demonstrate that both MH-DQN and MH-DDQN substantially outperform the heuristic baselines, and that the learned policies match or slightly exceed the SA acceptance ratio while achieving noticeably lower processing times, maintaining high deadline compliance (up to approximately 85% acceptance at heavy load), and reducing latency. MH-DDQN achieves the fastest convergence, greater stability, and slightly lower delays, highlighting its advantage for adaptive scheduling in complex MEC environments.