Beyond flips and rotations: evaluating NeRF and 3DGS-based synthetic view augmentation: A case study of detecting visual skinning damage on sweetpotato storage roots

The identification of skinning defects in sweetpotatoes is an essential yet intricate task in current packaging and grading procedures in production lines and workshops. The effectiveness of these processes is considerably influenced by the occurrence of skin damages which creates a need for fast and precise detection. The conventional method for inspecting sweetpotatoes skinning relies predominantly on manual processes, which is time and labor intensive. This approach's precision is inherently contingent upon the acuity and reliability of human visual observation, posing limitations in terms of consistency and accuracy. Consequently, automating the packaging process becomes challenging. To support, enhance, and automate the process of sweetpotato grading and packing, this study used YOLO models (YOLO v5, v8, v11) to detect common visual skinning areas in sweetpotatoes and to quantify the damage nondestructively in real time. To achieve this, 350 images of sweetpotatoes with skinned areas were obtained using an iPhone 11 followed by utilizing different data augmentation techniques such as geometric transformations and advanced photorealistic view synthesis generated by Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) to better identify, classify, and quantify these regions. Furthermore, the integration of GaussianEditor, a neural 3D editing tool, enables semantic modification of reconstructed scenes, such as background replacement, to diversify and enrich datasets. The effectiveness of these models was evaluated based on reconstruction fidelity and training efficiency for synthetic data augmentation using evaluation metrics including Peak Signal-to-Noise Ratio and Structural Similarity Index. YOLO models exhibited high precision (0.834), recall (0.818), and mAP mask @0.5 score (0.822) on identifying visual skinning damage on sweetpotato storage roots, proving the potential to be used for real time skinning damage detection. Our findings demonstrated that 3D-aware neural rendering and editing pipelines hold potential for augmenting vision-based agricultural datasets, enabling more robust agricultural and phenotyping tasks. • Sweetpotato grading needs robust defect detection for quality assurance. • 3DGS and NeRF enable synthetic multi-view data for YOLO training. • Synthetic views boost sweetpotato defect detection accuracy and robustness. • Augmentation expands pose and scale diversity beyond 2D image transforms. • Approach improves model generalization for agricultural quality inspection.