Space Grafted Velocity 3-D Boat Detection for Unmanned Surface Vessel via mmWave Radar and Camera

Recently, unmanned surface vessels (USVs) have played an increasingly important role in autonomous exploration, and boat detection is an important task for USVs. While most existing boat detection methods focus on 2-dimensional (2D) detection, 3-dimensional (3D) detection which provides valuable spatial direction for moving target estimation has not been studied in the boat detection field. However, 3D boat detection on water surfaces faces challenging problems, such as small sizes of detected targets and diverse moving directions. Considering that traditional LiDAR-based 3D boat detection methods require high hardware costs, we fuse millimeter wave (MMW) radar and high semantic camera to achieve low-cost and high-quality 3D boat detection. We propose a novel radar-camera fusion boat 3D detection model named RCBDet. The proposed RCBDet employs a new dual radar encoder and first introduces Doppler speed information from MMW radar into neural network to overcome sparse radar points. A new radar-camera attention module is designed to effectively combine camera features, radar spatial features and radar velocity features, encapsulating not only shape and semantic attributes but also spatial orientation information. In our collected boat 3D detection dataset, RCBDet achieves state-of-the-art accuracy compared to other single-modality baselines and radar-camera fusion baselines. Moreover, we conducted comprehensive ablation experiments to validate the efficacy of the designed modules. The experimental results demonstrated that the proposed radar-camera fusion model effectively fuses MMW radar features and camera features.