NLDETR-YOLO: A decision-making method for apple thinning period

• A method for effectively making decisions during apple thinning stage is proposed. • The apple thinning dataset by manual operations in natural scenes is established. • Sample imbalance and less local feature information are solved by model optimizing. • The proposed method shows excellent performance compared to the baseline model. Utilizing machine vision to accurately and swiftly remove surplus young fruit during the apple thinning phase is essential for managing crop load and estimating apple production. Fruit thinning operations help balance the nutrition of fruit trees, reduce the risk of tree disease, and prevent the phenomenon of biennial bearing. At this stage, young fruits are characterized by their small size, susceptibility to being obscured, similar colors, and indistinct details, making thinning decision-making difficult. To address these challenges and improve apple thinning decision-making in natural scenes, we developed the NLDETR-YOLO detection model. Firstly, the Efficient Hybrid Encoder from the RT-DETR model is integrated into the feature pyramid (Neck) of YOLOv5n, addressing the optimization difficulties and robustness issues in the YOLO detector that arise from the need for NMS post-processing. Secondly, we introduced the NLBlock attention mechanism, which allows the convolution operation to capture larger dependency relationships. Finally, the loss function to the Slide-loss function was modified to address the imbalance between simple and difficult samples. The experimental results indicate that the proposed method achieved precision (P), recall (R), and mean average precision (mAP) of 80.6 %, 81.2 %, and 86.9 % respectively without overfitting. These represent improvements of 1.9 %, 0.7 %, and 1.4 % compared to the baseline network, with minimal changes to parameters, GFLOPS, and model size. The enhanced NLDETR-YOLO algorithm demonstrates high robustness and real-time detection performance, enabling rapid and accurate thinning decision-making and providing effective support for the automated management of apples during the thinning period in natural environments.