Semi-MeshSeg: A semi-supervised semantic segmentation network for large-scale urban textured meshes using all pseudo-labels

Urban mesh data comprises large-scale textured meshes representing outdoor urban environments. Semantic segmentation of urban meshes is becoming increasingly important in urban analysis. However, many existing studies predominantly employ fully supervised methods, which typically require substantial facet-level annotations that are challenging to acquire in practice. To address this issue, we propose a novel semi-supervised learning approach, Semi-MeshSeg, to tackle the problem of insufficiently labeled data. Urban meshes are irregular and susceptible to distortion, leading to blurred boundaries and surface features. To solve this challenge, we introduce a novel semi-supervised method for 3D mesh semantic segmentation, capable of leveraging all pseudo-labels, rather than relying solely on a small subset of high-confidence ones, to more effectively capture the inherent variability and irregularity of urban meshes. Given that pseudo-labels, particularly low-confidence ones, often contain substantial noise, we design dual-stream perturbation branches, a noise reduction loss, and a distribution alignment loss to mitigate this noise. The dual-stream perturbation branches enhance the semantic segmentation performance of the model by exploring extensive three-dimensional (3D) perturbation spaces, while the noise reduction loss and distribution alignment loss are designed to enhance the quality of noisy pseudo-labels. Compared to state-of-the-art (SOTA) methods, our approach achieves superior performance with only one tile of labeled data, improving mIoU by at least 3.0% on the SUM dataset and 4.5% on the seMet dataset, demonstrating its effectiveness under limited labeled data conditions. • Semi-MeshSeg is the first semi-supervised method for urban meshes semantic segmentation. • Semi-MeshSeg leverages all pseudo-labels during training. • The noise reduction and distribution alignment losses reduce noise in the pseudo-labels. • The dual-stream perturbation branches explore extensive perturbation spaces.