Zero-Shot Prompt-Based Partial 3D Point Cloud Creation of the Specified Object From an Unlabeled 2D Image

This project aims to generate RGBD point clouds from specific objects in unlabeled 2D images using zero-shot, prompt-based methods. By integrating advanced machine learning models for object detection, image segmentation, and depth estimation, it creates accurate 3D models from 2D data. Leveraging the recent advancements in computer vision, this approach utilizes textual prompts to perform tasks without prior explicit training. The model inputs a 2D image and outputs a partial RGBD point cloud, effectively transforming 2D images into 3D representations. The primary goal is to demonstrate the feasibility and effectiveness of this zero-shot framework. Objectives include achieving effective detection in prompt-based object detection, high accuracy in segmentation, and creating high-quality depth maps. The integration of these technologies aims to produce detailed and reliable 3D outputs suitable for various real-world applications. The generated 3D models have significant potential to build new 3D models and expand training datasets in areas where data is limited - by synthetically generating data with 3D models of underrepresented objects, thereby reducing bias in machine learning applications.