Picking point identification and localization method based on swin-transformer for high-quality tea

In the nature scene, because of the high degree of similarity between the background and the tea buds, as well as the different growth postures of the tea buds, finding and precisely identifying the picking point is challenging. To solve these issues, this paper proposes a precise way to find the best picking point for tea buds by combining traditional algorithms with Swin-Transformer-based target detection and semantic segmentation algorithms, namely SORC-SFT. Firstly, an improved target detection algorithm, Swin-Oriented R-CNN (SORC), is used to realize the recognition of four types of high-quality tea. The mean Average Precision (mAP) of the four categories was 82.3% after replacing the feature fusion network FPN with PAFPN and adding the Coordinate Attention (CA) mechanism. Secondly, the corresponding segmentation mask of the four recognized categories is obtained by adding Semask, Feature Alignment Module (FAM), and Feature Selection Module (FSM) to the improved semantic segmentation algorithm Semask-Fa-Transformer (SFT). The mean Intersection over Union (mIoU) of the semantic segmentation algorithm for each category is 89.83%, 91.97%, 88.85%, and 89.68%, respectively. Finally, the morphology of different categories of tea buds is analyzed, and the traditional algorithm is used to realize the accurate localization of the identified tea buds. For the four tested categories, the proportion of correct samples in locating picking points is 96.18%, 91.28%, 93.85%, and 90.58%, respectively. The experimental results show that, out of all the algorithms, the proposed picking point identification and localization approach has the best performance and will make a strong contribution to the accurate identification of tea leaves during the intelligent picking process.