Millimeter-Wave Radar and Camera Fusion for Multiscenario Object Detection on USVs

Accurate object detection is fundamental for unmanned surface vehicles (USVs) to achieve intelligent perception. This article proposes an object detection network that integrates millimeter-wave radar and a camera. The method utilizes the complementary advantages of millimeter-wave radar and camera data modalities to realize multiscenario object detection for USVs applications. To address the drawback of sparse point clouds in millimeter-wave radar and improve the suboptimal performance of the camera in adverse weather conditions and small object detection, as well as to effectively utilize the features of both millimeter-wave radar and camera, a multisensor deep learning fusion object detection network [fusion mixture with AFPN (FMA)-fully convolutional one-stage (FCOS)] is proposed. To validate the effectiveness of FMA-FCOS, training, and testing are conducted on the multiscenario vessel dataset collected specifically for this study and the nuScenes dataset. In comparison with methods solely relying on a camera, such as the original FCOS object detection framework and YOLOv9, as well as other fusion methodologies combining camera and radar, the results demonstrate that FMA-FCOS delivers notable advantages, achieving a superior or comparable detection accuracy in the datasets.