Multi-Map Fusion for Vision-and-Language Navigation in Continuous Environment

Visual Language Navigation (VLN) is a task that challenges agents to navigate through unseen environments solely based on language and visual supervision. In the recently proposed continuous setting (Continuous VLN), agents must navigate within a free 3D space. To achieve better performance in Continuous VLN, this paper introduces a new navigation framework. In this framework, the agent observes the environment in a continuous setting and constructs grid maps and topological maps based on acquired information, which are respectively used for short-term reasoning and long-term reasoning. Finally, through hybrid action fusion, the navigation scores from both maps are combined using an algorithm to determine the agent's final action space. This paper elaborates on how to construct grid maps and topological maps and demonstrates through ablation experiments the improvement in the agent's navigation success rate with the introduction of map representations. Furthermore, to enhance the model's understanding of multimodal information and utilization of map information, three pretraining tasks are employed during the pretraining phase. Ultimately, experimental results on the Habitat simulator validate the effectiveness of the proposed model, as demonstrated by its excellent performance on the VLN-CE dataset.