Robotic self-supervised local view planning for efficient and scalable 3D plant reconstruction across varying plant sizes

• SSL-Local-NBV enables self-supervised local view planning for 3D plant reconstruction. • It improves trajectory efficiency over global NBV methods. • It scales effectively across varying plant sizes. • It generalizes well in both simulation and real-world settings. Accurate perception in complex agricultural environments is challenging due to significant plant occlusion, primarily from leaves, which hinder data collection and increase uncertainty in robotic operations. Deep-learning-based Next-Best-View (DL-NBV) methods address this by using neural networks to predict information gain (IG) for potential camera views and actively repositioning the camera to maximize data collection with minimal views. However, training DL-NBV models requires extensive IG-labeled data. A self-supervised learning-based NBV method, SSL-Global-NBV, enables robots to collect training data autonomously and improve themselves for global NBV planning. Despite its advantages, SSL-Global-NBV has two key limitations: (1) it requires a fixed number of views, limiting scalability across different plant sizes, and (2) it selects views globally, resulting in inefficient view transitions across the entire view space, reducing trajectory efficiency. To overcome these limitations, this paper introduces SSL-Local-NBV, which incorporates local view planning for scalable and efficient view selection. To prevent redundant visits to the same views, a View Trajectory Network (VTN) was proposed to memorize the view trajectory information of visited views. Comprehensive evaluations in simulation and real-world plant reconstruction demonstrated that SSL-Local-NBV reduced trajectory distance by 56%–70% per reconstruction cycle, achieving 267%–300% higher trajectory efficiency than global NBV methods. Compared to SSL-Global-NBV, SSL-Local-NBV improved plant reconstruction efficiency by 5.2% across varying plant sizes, demonstrating greater scalability. For real plants, SSL-Local-NBV achieved over 80% reconstruction, confirming its feasibility in practical applications. Notably, SSL-Local-NBV fully automated training through self-supervised learning, enabling continuous and lifelong robotic learning.