DRFSL: Deep Reinforced Federated Split Learning for Multi-Modal Beamforming in IoV

In Vehicle-to-Everything (V2X) communication, advanced beamforming techniques address signal attenuation caused by mmWave, which provides high bandwidth and low latency. Multi-modal beamforming using Federated Learning (FL) can leverage resources like GPS, Lidar, and image data, significantly accelerating beam searching while enhancing data privacy. The heterogeneity of vehicles, however, affects the availability of computing resources for training machine learning models. Moreover, the multi-modal fusion network may contain billions of parameters, leading to extended training time for FL. To address these challenges, this paper proposes a novel Deep Reinforced Federated Split Learning framework (DRFSL) tailored for multi-modal beamforming with different sub-model architectures. DRFSL efficiently utilizes MEC computing and adapts the collaborative and distributed training to dynamic network conditions and system heterogeneity by incorporating deep reinforcement learning and split learning with FL. Experimental evaluation using real-world datasets demonstrates that DRFSL minimizes average training time by 49.45% and inference time by 24.43% and can achieve higher accuracy within the same timeframe compared to the existing FLASH framework.