Cloud-Driven Framework for Semantic Object Mapping in Indoor Environments

Semantic mapping of the environment plays a crucial role in determining how a robot interacts with its surroundings. Most robots, however, lack sufficient computational capabilities to process and perform inference for environmental understanding. This project introduces a pipeline for generating a semantic object list of an indoor environment. The proposed algorithm utilizes cloud-based GPU instances for segmentation and cloud-based multimodal large language models for scene and area recognition. The pipeline’s latency is evaluated to ensure its suitability for mobile robot applications. The developed algorithm effectively generates semantic object lists that could facilitate robot operation in an indoor environment.