Accurate organ segmentation and phenotype extraction of tomato plants based on deep learning and clustering algorithm

• In this study, we take tomato as an example and propose an improved deep learning combined with clustering algorithm for plant point cloud segmentation. First, a multi-temporal tomato point cloud dataset is constructed by combining multi-view image sequences with neural radiation field (NeRF), and preprocessing and labeling are completed; second, single channel attention (SCA) and global feature aggregation module (GFA) are introduced into the PointNet++ model, respectively, to construct the TomatoSegNet model, which improves the tomato dataset's semantic segmentation performance, while the edge filter was added to the DBSCAN algorithm to improve it and enhance the instance segmentation performance of canopy leaves; finally, a total of six phenotypic parameters were extracted based on the segmented organs. The experimental results show that the TomatoSegNet model has an average precision (mP) of 97.82%, an average recall (mR) of 98.62%, an average F1 score (mF1) of 97.97%, an average intersection and merger ratio (mIoU) of 96.84%, and an overall accuracy (OA) of 94.22% in the tomato dataset, which proves that the use of semantic segmentation algorithms feasibility of stem and leaf segmentation; the improved DBSCAN algorithm achieved an instance segmentation accuracy of 96.03% for leaves, which improved the segmentation accuracy of overlapping leaves; the coefficients of determination between the measured and calculated values of the six phenotypic parameters (plant height, stem thickness, leaf inclination, leaf length, leaf width, and leaf area) were 0.983, 0.903, 0.916, 0.962, 0.951, and 0.978. The method proposed in this study realizes the accurate segmentation and extraction of phenotypic parameters from the 3D point cloud of plants, which provides a valuable reference for automated phenotypic analysis of plants.. In plant phenotyping research, accurate organ segmentation and phenotype extraction is the key to accelerate the process of big data analysis and intelligent breeding.In this study, we take tomato as an example and propose an improved deep learning combined with clustering algorithm for plant point cloud segmentation. First, a multi-temporal tomato point cloud dataset is constructed by combining multi-view image sequences with neural radiation field (NeRF), and preprocessing and labeling are completed; second, single channel attention (SCA) and global feature aggregation module (GFA) are introduced into the PointNet++ model, respectively, to construct the TomatoSegNet model, which improves the tomato dataset's semantic segmentation performance, while the edge filter was added to the DBSCAN algorithm to improve it and enhance the instance segmentation performance of canopy leaves; finally, a total of six phenotypic parameters were extracted based on the segmented organs. The experimental results show that the TomatoSegNet model has an average precision (mP) of 97.82%, an average recall (mR) of 98.62%, an average F1 score (mF1) of 97.97%, an average intersection and merger ratio (mIoU) of 96.84%, and an overall accuracy (OA) of 94.22% in the tomato dataset, which proves that the use of semantic segmentation algorithms feasibility of stem and leaf segmentation; the improved DBSCAN algorithm achieved an instance segmentation accuracy of 96.03% for leaves, which improved the segmentation accuracy of overlapping leaves; the coefficients of determination between the measured and calculated values of the six phenotypic parameters (plant height, stem thickness, leaf inclination, leaf length, leaf width, and leaf area) were 0.983, 0.903, 0.916, 0.962, 0.951, and 0.978. The method proposed in this study realizes the accurate segmentation and extraction of phenotypic parameters from the 3D point cloud of plants, which provides a valuable reference for automated phenotypic analysis of plants.