3D-NOD: 3D new organ detection in plant growth by a spatiotemporal point cloud deep segmentation framework

Automatic plant growth monitoring is an important task in modern agriculture for maintaining high crop yield and boosting the breeding procedure. The advancement of 3D sensing technology has made 3D point clouds to be a better data form on presenting plant growth than images, as the new organs are easier identified in 3D space and the occluded organs in 2D can also be conveniently separated in 3D. Despite the attractive characteristics, analysis on 3D data can be quite challenging. We present 3D-NOD, a framework to detect new organs from time-series 3D plant data by spatiotemporal point cloud deep semantic segmentation. The design of 3D-NOD framework drew inspiration from how a well-experienced human utilizes spatiotemporal information to identify growing buds from a plant at two different growth stages. In the training phase, by introducing the Backward & Forward Labeling, the Registration & Mix-up, and the Humanoid Data Augmentation step, our backbone network can be trained to recognize growth events with organ correlation from both temporal and spatial domains. In testing, 3D-NOD has shown better sensitivity at segmenting new organs against the conventional way of using a network to conduct direct semantic segmentation. On a time-series dataset containing multiple species, Our method reached a mean F1-measure at 88.13 ​% and a mean IoU at 80.68 ​% on detecting both new and old organs with the DGCNN backbone.