Towards Autonomous Robotic Grasping: 3D Point Cloud Analysis with Stereo Cameras in Unpredictable Environments

Current robotic arm systems typically depend on pretrained models or function within controlled environments using known positions or fiducial markers for object manipulation. Although these methods achieve high accuracy, they face limitations due to lengthy training processes and the need for carefully controlled setups. This research introduces a methodology that leverages stereo cameras to develop a generalized algorithm aimed at overcoming these constraints and enhancing robotic manipulation in uncontrolled environments. The proposed solution utilizes an Intel RealSense stereo camera in conjunction with YOLO segmentation models for initial object identification. It combines segmentation meshes with the camera's depth values to create a detailed three-dimensional point cloud of the target object's surface. This point cloud is analyzed through Random Sample Consensus (RANSAC) and geometry-based feature extraction techniques, which allow the algorithm to approximate the object's shape as a combination of fundamental 3D forms, such as cuboids, cylinders, and spheres. A grasp point is then estimated by identifying suitable surfaces for grasping with a two-pronged gripper, followed by calculating an optimal approach angle and position kinematics for the robotic arm. This innovative approach enables robotic arm systems to autonomously adapt to various objects in uncontrolled environments, significantly expanding their practical applications. For example, the methodology could support modular robotic systems designed for sample collection in extraterrestrial missions, where adaptability and autonomous operations are essential. Ultimately, this research aims to enhance the efficiency and capabilities of robotic systems in dynamic and unpredictable settings.