Deep Learning-Based Selective Feature Fusion for Litchi Fruit Detection Using Multimodal UAV Sensor Measurements

In the field of precision agriculture, accurate crop detection is crucial for crop yield estimation, and health monitoring using photogrammetric measurements. Achieving high precision requires advance object detection models and multiscale feature fusion. This article addresses key research gaps in litchi crop monitoring, including the lack of a suitable dataset for litchi detection in natural environment and the limitations of conventional deep learning models in handling challenges such as occlusion, overlapping, and background complexities. First, we prepare high-resolution litchi dataset called “UAVLitchi” of 5000 images that include both RGB and multispectral images and next, we propose a selective feature fusion (SFF)-based architecture for litchi detection. By utilizing both RGB and multispectral images, this architecture effectively mitigates the challenges of visual detection arising from the complex cluster growth structure of litchis, offering a robust solution for accurate detection. The integration of SFF within a dual-channel mask-region based convolutional neural network (Mask-RCNN) leading to significant improvements in feature extraction for litchi detection. Experimental results demonstrate impressive performance, achieving an mean average precession (mAP50) of 94.65%, mAP75 of 89.23%, recall of 90.16%, and F1-score of 91.44%.