Depth-Guided Texture Diffusion for Image Semantic Segmentation

Depth information provides valuable insights into the 3D structure, especially the outline of objects, which can be utilized to enhance semantic segmentation. However, a naive fusion of depth information can disrupt features and compromise accuracy due to the gap between depth and RGB modalities. In this work, we introduce a depth-guided texture diffusion approach that effectively tackles the outlined challenge. Our method extracts low-level features from edges and textures to create a texture image. This image is then selectively diffused across the depth map, enhancing structural information vital for precisely extracting object outlines. By integrating this enhanced depth map with the original RGB image into a joint feature embedding, our method effectively bridges the disparity between depth and RGB modalities, enabling more accurate semantic segmentation. We conduct comprehensive experiments on diverse, widely-used datasets covering various semantic segmentation tasks, including Camouflaged Object Detection (COD), Salient Object Detection (SOD), and indoor semantic segmentation. With source-free estimated depth or depth captured by depth cameras, our method consistently outperforms existing baselines and achieves new state-of-the-art results, demonstrating the effectiveness of our depth-guided texture diffusion for image semantic segmentation. The source code and datasets are publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Wistzz/Texture-Diffusion.git</uri>.