SAM-Assisted 3D Reconstruction: A Novel Framework for Unseen Object Modeling in Unknown Environments

This paper tackles the challenging problem of Three-dimensional (3D) reconstruction for objects in unknown environments, which poses significant demands on object recognition and segmentation. We present an innovative framework for high-fidelity 3D point cloud reconstruction of previously unseen objects utilizing the Intel RealSense D435i cameras. The proposed framework comprises two main components: point cloud extraction and multi-view reconstruction. Specifically, the Segment Anything Model (SAM) is introduced for object point cloud segmentation, and the Point-to-Plane Iterative Closest Point (ICP) algorithm is used for multi-view registration to achieve precise and detailed 3D reconstruction from multiple view images. Through experimental validation, our proposed framework not only accomplishes point cloud reconstruction of objects in unknown environments, but also demonstrates superior accuracy compared to conventional multi-view reconstruction methods. Comprehensive experiments on a variety of tabletop objects substantiate the efficacy of our proposed approach, showcasing its performance in 3D point cloud reconstruction within unknown environments.