Automated 3D Segmentation of Plant Organs via the Plant-MAE: A Self-Supervised Learning Framework

Reliable and automated three-dimensional segmentation of plant organs is essential for extracting phenotypic traits at the organ level. However, existing methods for plant organ segmentation predominantly rely on fully supervised learning, which still necessitates extensive point-by-point annotated datasets and fails to overcome the challenges associated with annotating plant point cloud data. In recent years, self-supervised learning-based point cloud segmentation methods have garnered widespread attention in both industry and academia because of their potential to alleviate the difficulties of point cloud data annotation to some extent. In this study, the paradigm of self-supervised learning is innovatively applied to the field of plant phenotyping through the development of the Plant-MAE, a self-supervised learning-based point cloud segmentation framework. The innovations of the Plant-MAE include a kernel-based point convolution embedding module and a multiangle feature extraction block (MAFEB) based on attention mechanisms. To validate the effectiveness of the model, extensive experiments were conducted on multiple point cloud datasets, which achieved competitive performance, with average precision, recall, F1 score, and IoU values of 92.08 ​%, 88.50 ​%, 89.80 ​%, and 84.03 ​%, respectively. The Plant-MAE outperforms advanced deep learning networks, including PointNet++, point transformer, and Point-M2AE, achieving average improvements of at least 0.53 ​%, 1.36 ​%, 0.88 ​%, and 2.38 ​% in precision, recall, F1 score, and IoU, respectively. Additionally, on the Pheno4D dataset, only half of the training data were necessary for fine-tuning to achieve performance comparable to that of the point transformer and PointNet++. This study provides technical support for the estimation of crop phenotypic parameters, thereby advancing the development of modern smart agriculture.