Development of autonomous navigation system based on neural network and visual servoing for row-crop tracking in vegetable greenhouses

• The network directly outputting effective areas of the navigation line is designed. • Adding an attention mechanism in the backbone enhances the overall task performance. • The control scheme is only based on line and point features extracted from the image. Accurate navigation line localization strategy and robust path tracking control scheme are two key techniques for advancing precision agriculture and enhancing the sustainability of smart farms. In this paper, a superior and cost-effective autonomous navigation system for a visual servoing-based agricultural vehicle in the greenhouse scenario with a distinct layout of two crop rows in each ridge is presented. First, based on image segmentation techniques, a neural network model named DGLNet integrating recognition of crop rows and prediction of the effective area of the navigation line is designed. The linear equation of the navigation line we required is obtained through post-processing operations of linear fitting completed in the effective area. According to experimental results, directly teaching the neural network to learn navigation line localization features utilizes the semantic and spatial information in the feature map to establish a stronger relationship between crop rows and the navigation line, which greatly improves the generalization of the navigation line localization in the case of large-angle yaw scenarios without additional training datasets. In addition, the performance of the network is significantly enhanced by adding an attention mechanism in the backbone based on U-Net. Subsequently, a dataset of cabbage vegetable images covering a wide range of complexities in the greenhouse scenario is created, and is available to support future academic research now. Further, a path tracking control scheme based on point-feature and line-feature is proposed, which can control the agricultural vehicle using the information from camera images without the need to perform explicit localization or maintain a global map. Finally, extensive field experiments have been carried out to show that this autonomous navigation system can preserve the stability of the vehicle and ensure timely convergence of deviations in different complex scenarios, which indicates that this control scheme is reliable and robust. Meanwhile, benefit from the efficiency of this recognition algorithm and the unnecessity of the complete representation of the path in the control scheme, the system has high real-time performance in every scenario.