Embodied Navigation in Unknown Environments with Implicit Scene Memory and Target-aware Memory Retrieval

Neural Radiance Fields (NeRF) have demonstrated significant potential in providing fine and dense scene representations. This paper intend to leverage NeRF as a memory structure for storing scene cues and to explore its potential in robotic navigation tasks, thereby enabling robots to achieve reliable and efficient navigation in unknown environments. During navigation, scene features from historical observations are stored online in the NeRF memory. Concurrently, exploiting the implicit characteristics of NeRF, a differentiable memory retrieval mechanism called Dual Space Attention is designed to extract target-relevant scene cues from the NeRF structure, supporting long-term optimized behavior patterns. Additionally, to address the significant noise challenges due to the lack of environmental priors when using implicit NeRF memory in unknown scenes, an uncertainty masking operation is introduced during memory retrieval to eliminate low signal-to-noise ratio information and potentially enhance exploration behavior. Results from photorealistic simulations and real-world demonstrations show that the proposed navigation system exhibits obvious target search and orientation behavior patterns, outperforming typical baselines in terms of performance and efficiency. Overall, the proposed end-to-end navigation pipeline extends the application of NeRF technology in control domain tasks, offering new possibilities for the advancement of intelligent robots and Embodied AI.