Tobacco stem and leaf segmentation and phenotypic parameter extraction based on the improved point cloud segmentation network PE-KPConv

• Combined with multi-view images collected by drones, an open-source tool ContextCapture and the YOLOv11n algorithm are used to construct a tobacco point cloud dataset. • We propose PE-KPConv, a novel framework for accurate stem and leaf segmentation in tobacco plant point clouds. This model integrates the PCCPAFE and EAM modules and employs a multi-scale lateral output architecture. • Four phenotypic parameters (plant height, leaf length, leaf width, and leaf area) were extracted using an optimized DBSCAN clustering method, and these parameters were very close to the true values measured on the ground. Plant phenotypic analysis is crucial for understanding crop growth and realizing smart agriculture. In recent years, the segmentation of plant organs based on 3D point clouds has been of great significance for extracting plant phenotypic parameters. However, although Kernel Point Convolution (KPConv) shows excellent performance in point cloud segmentation due to its geometric adaptability, it is vulnerable to occlusion interference because of the lack of a dynamic attention mechanism, and the utilization of context information in the decoding stage is insufficient, making it difficult to meet the requirements for fine segmentation of leaf organs. Therefore, we explored the feature extraction capabilities of KPConv and proposed a segmentation network named PE-KPConv based on multi-scale attention and lateral feature reuse. First, a tobacco point cloud dataset was constructed by combining multi-view images collected by drones and the YOLOv11n algorithm. Then, in the encoding stage, the PCCPAFE module was designed, which adopted a multi-scale dynamic aggregation strategy to enhance the geometric features at the connection between the leaf surface and the stem. In the decoding stage, the EAM module was designed to alleviate the feature confusion caused by leaf occlusion by expanding the receptive field. Additionally, a multi-scale lateral output structure was built to achieve the reuse of upsampled features at 8×/4×/2× to retain edge details. Experiments on the tobacco dataset showed that the performance metrics mIoU, mACC, and F1 of the PE-KPConv network reached 92.35%, 93.36%, and 97.40% respectively. Finally, based on the point cloud segmentation results, four phenotypic parameters (plant height, leaf length, leaf width, and leaf area) were extracted through an optimized DBSCAN clustering method. These parameters were very close to the real values measured on the ground, and the corresponding measured values were all above 0.96. Moreover, ablation studies and generalization experiments also demonstrated that the PE-KPConv network is robust and widely applicable. This method realizes the automated data analysis from plant 3D point clouds to phenotypic parameters, providing strong technical support for phenotypic research in smart tobacco agriculture.