PointNeXt-DBSCAN: a hybrid point cloud deep learning framework for multi-stage cotton leaf instance segmentation

This study addresses the challenge of organ-level instance segmentation in cotton point clouds, which arises from significant morphological variations and leaf occlusion across growth stages. To achieve high-precision leaf extraction, a hybrid framework integrating PointNeXt and DBSCAN is proposed. A dataset containing 1,065 cotton plants from seedling to boll-opening stages was constructed via multi-view image reconstruction and augmented through random rotation and scaling. Methodologically, a two-stage pipeline was designed: semantic segmentation was first performed using the PointNeXt network, where its residual MLP blocks enhanced edge and local feature learning; instance segmentation was then conducted by applying density-adaptive DBSCAN clustering to the semantic results, effectively mitigating over-segmentation in emerging leaves. Experimental results indicate that the semantic segmentation achieved an mIoU of 0.9846, representing a 7.2% improvement over PointNet++. The subsequent instance segmentation attained an ARI of 0.983, reduced the over-segmentation rate by 63%, and maintained an error below 3% for leaves smaller than 5 cm². The framework provides reliable technical support for the automated extraction of key phenotypic traits such as leaf area index and leaf inclination distribution.