Pattern-Aware Indoor Semantic Segmentation Based on Visual Optimization Learning

Semantic segmentation of indoor scenes plays a pivotal role in various applications such as indoor navigation systems, creating detailed and accurate indoor maps, and optimizing route planning for both humans and robots. However, effectively integrating RGB (color) and HHA (depth) images for indoor semantic segmentation poses both promise and challenge due to the diverse textures and structures present in indoor environments, coupled with the varying significance of different modalities. Traditionally, tasks like semantic segmentation and depth estimation have been treated as separate entities in computer vision. However, recent advancements in research have explored the potential benefits of combining these tasks into a unified framework. The rationale behind this exploration lies in the inherent correlation between semantic understanding and depth perception, suggesting that integrating these tasks could lead to more accurate estimations. Our approach diverges from conventional segmentation methods by proposing the integration of two neural network models to achieve precise semantic segmentation specifically tailored for mobile robot mapping applications. Initially, we employ a model to identify salient objects within the scene, which are then delineated in grayscale. Subsequently, the contours of these salient objects, along with their semantic labels, are projected onto the corresponding RGB image, resulting in the segmentation of important objects within the indoor environment. By focusing on salient objects rather than considering the entirety of the scene, our method streamlines the segmentation process, simplifying the consideration of background elements. This targeted approach is particularly advantageous in scenarios where efficient and accurate segmentation of relevant objects is paramount, such as in robotic navigation tasks within complex indoor environments.