WLMMF-Wireless Multimodal Fusion of mm-Wave Radar and Vision for Real-Time Human Action Recognition (RT-HAR) in Mobile Robotics (MR)

Human Action Recognition (HAR) is essential for enabling mobile robots to interact intelligently and safely in human-centric environments. This study introduces a wireless-aware multimodal fusion framework that integrates millimeter-wave (mmWave) radar with vision sensing for real-time HAR, ensuring robustness even under fluctuating wireless and environmental conditions. Unlike prior multimodal HAR frameworks that assume stable connectivity, our design explicitly embeds wireless channel-state feedback into the fusion process, ensuring real-time adaptability under dynamic communication conditions. The radar modality captures motion kinematics via range–Doppler and point-cloud signatures, while the camera provides spatial and appearance cues. Latency on the prototype has been kept below 50 milliseconds, and the proper integration of these heterogeneous features is additionally safeguarded through a lightweight deep-learning pipeline using adaptive fusion weighting. All the experiment simulation Outcome results showed that the proposed system obtains recognition accuracy values of up to 92% and an F1-score of 0.91 which actually outperforms both vision-only having 84% accuracy along with radar-only with 86% accuracy baselines. The results prove that wireless–vision fusion is possible for deploying real-world mobile robots in which robustness to environmental uncertainty becomes very important. The work also establishes the framework of wireless sensing in supporting context-aware human–robot interaction.