MA-DDSCNet: multi-agent dynamic diffusion semantic communication network for task-oriented multi-modal fusion in autonomous driving

In bandwidth-limited and time-varying vehicle–road–cloud cooperative autonomous driving scenarios, real-time transmission and joint inference of high-dimensional multimodal perception data are simultaneously constrained by latency, reliability, and energy consumption. To address these challenges, this article proposes a task-oriented multimodal fusion framework named Multi-Agent Dynamic Diffusion Semantic Communication Network (MA-DDSCNet). On the vehicle side, we design a Task-Guided Multi-Modal Semantic Encoder (TG-MMSE) that performs spatio-temporal alignment, complementary memory gating, and differentiable discrete quantization to compress heterogeneous perception streams from cameras, Light Detection and Ranging (LiDAR), and vehicular state into task-weighted discrete token sequences. A hierarchical distillation scheme is further employed to maintain a unified semantic coordinate system across vehicles, Road Side Units (RSUs), and the cloud. On the communication side, a hierarchical controllable diffusion mechanism adaptively adjusts diffusion noise and time steps according to the importance of object detection, trajectory prediction, and motion planning tasks, as well as link-specific bandwidth budgets. A multi-agent deep scheduler enables collaborative utilization of communication resources among the cloud, RSUs, and vehicles, while an iterative joint semantic decoding and consistency calibration algorithm feeds residuals back into a global memory matrix to suppress semantic drift and yield isomorphic semantic representations at all three layers. Furthermore, we construct a learnable uncertainty-driven multi-objective loss function, combined with a gradient projection strategy, to achieve end-to-end joint optimization of detection, prediction, and planning within a single training loop. Simulation results demonstrate that, compared with baseline methods, MA-DDSCNet achieves average gains of 9.6–18.4% in mean Average Precision (mAP), Average Displacement Error (ADE), Final Displacement Error (FDE), Effective Bit Rate (EBR), and planning safety rate, while reducing the 95th-percentile end-to-end latency to 63 ms, indicating that the proposed framework can significantly enhance the overall performance of semantic perception tasks in complex vehicular networks.