Oil palm tree detection in UAV imagery using an enhanced RetinaNet

• Automatic detection of oil palm trees in unmanned aerial vehicle imagery. • Detecting young and mature oil palm trees using a deep learning model. • Comparison of state-of-the-art methods for oil palm tree detection. Accurate inventory management of oil palm trees is crucial for optimizing yield and monitoring the health and growth of plantations. However, detecting and counting oil palm trees, particularly young trees that blend into complex environments, presents significant challenges for deep learning models. While current methods perform well in detecting mature oil palm trees, they often struggle to generalize across the diverse variations found in both young and mature trees. In this study, we propose an enhanced RetinaNet model that incorporates deformable convolutions into the ResNet-50 backbone, deeper feature pyramid layers, and an intersection-over-union-aware branch in a multi-head configuration to improve detection performance. The model was evaluated using a diverse dataset of unmanned aerial vehicle imagery from multiple regions, encompassing oil palm and coconut trees, as well as banana plants. To refine detection, confidence thresholding and non-maximum suppression were applied during inference, filtering out low-confidence predictions and eliminating duplicate detections. Experimental results demonstrate that our method outperforms state-of-the-art models, achieving F1-scores of 0.947 and 0.902 for single- and dual-species detection tasks, respectively, surpassing existing approaches by 1.5–6.3%. These findings highlight the model’s ability to accurately detect oil palm trees, particularly young ones in complex backgrounds, offering a reliable solution to support sustainable agriculture and improved land management.