Query-based Object-aware Mapping for On-device Visual Language Mapping and Navigation

Recent advancements in artificial intelligence, particularly in multimodal models such as large language models (LLMs) and visual language models (VLMs), have enabled robots perform zero-shot learning, allowing them to efficiently complete a variety of tasks. This paper introduces DMAP: a query-based object-aware mapping method that utilizes VLMs to produce lightweight and effective navigation maps for edge robots. By dividing, encoding and saving keyframes with observed voxels, our method minimizes computational complexity, which allows real-time object aware mapping and language-driven navigation on low-powered devices, such as the Nvidia Jetson Orin Nano. This approach significantly reduces both map creation time and storage requirements, supporting zero-shot navigation without the need for prior object learning.