Multimodal UAV Target Detection Method Based on Acousto-Optical Hybridization

Urban unmanned aerial vehicle (UAV) surveillance faces significant obstacles due to visual obstructions, inadequate lighting, small target dimensions, and acoustic signal interference caused by environmental noise and multipath propagation. To address these issues, this study proposes a multimodal detection framework that integrates an efficient YOLOv11-based visual detection module—trained on a comprehensive dataset containing over 50,000 UAV images—with a Capon beamforming-based acoustic imaging system using a 144-element spiral-arm microphone array. Adaptive compensation strategies are implemented to improve the robustness of each sensing modality, while detections results are validated through intersection-over-union and angular deviation metrics. The angular validation is accomplished by mapping acoustic direction-of-arrival estimations onto the camera image plane using established calibration parameters. Experimental evaluation reveals that the fusion system achieves outstanding performance under optimal conditions, exceeding 99% accuracy. However, its principal advantage becomes evident in challenging environments where individual modalities exhibit considerable limitations. The fusion approach demonstrates substantial performance improvements across three critical scenarios. In low-light conditions, the fusion system achieves 78% accuracy, significantly outperforming vision-only methods which attain only 25% accuracy. Under occlusion scenarios, the fusion system maintains 99% accuracy while vision-only performance drops dramatically to 9.75%, though acoustic-only detection remains highly effective at 99%. In multi-target detection scenarios, the fusion system reaches 96.8% accuracy, bridging the performance gap between vision-only systems at 99% and acoustic-only systems at 54%, where acoustic intensity variations limit detection capability. These experimental findings validate the effectiveness of the complementary fusion strategy and establish the system’s practical value for urban airspace monitoring applications.