Fine-Structured Learning for Semantic Map Representation

Building the accurate semantic map is crucial for embodied agents to navigate in the unknown environment. In such context, the semantic mapping task, building allocentric top-down map from egocentric observations such as RGBD camera and its pose, has been advanced rapidly. In this paper, we propose novel CNN decoder architecture and fine-structured loss to train more detail semantic and spatial information of map. Inspired by the modern 3D reconstruction algorithm like NeRF, we adopted the L2 loss when training the map data. This paper evaluates the proposed framework on the Matterport3D benchmark, achieving competitive performance to the baseline, SMNet, and analysis the map segmentation results.