RT-BMTR: A Bilateral Hybrid Backbone Network for Crop and Weed Detection in Complex Agricultural Scenarios

For modern agricultural management, the accuracy of plant identification is crucial. However, the task becomes challenging because crops and weeds at early growth stages often exhibit similar color, leaf morphology, and texture in two-dimensional images captured under field conditions, despite their clear biological differences in terms of botanical species, root systems, and phenological characteristics. Furthermore, computing hardware in the field also has strict limits. Therefore, we developed the RT-BMTR network to handle these physical constraints. Within this architecture, image data is processed through a bilateral hybrid backbone named Bi-HMB. The DSFM captures small local details, and MambaVision understands the broader background information. Then, these features are fused by RepNCSPELAN4. We adopted this structure to reduce redundant calculations. Next, the model determines its bounding boxes using the Inner-ShapeIoU loss function. This geometric constraint improves the detection of small targets. When evaluated on the CropAndWeed dataset, our model achieved an average precision (AP) at IoU threshold 0.5 (AP50) of 68.1%, AP75 of 54.8%, and a mean AP averaged over IoU thresholds from 0.5 to 0.95 (AP50–95) of 50.9%. Detection precision recorded 26.5% for small objects, 44.7% for moderate ones, and with 59.3% for large objects. Rates for the first two categories saw enhancements of 16.2% and 4.6%. Overall, our modified model outperforms the original RT-DETR baseline. We also shrank the overall parameter count by 30.1%, alongside a 4.2% decrease in computational demand.