Remote Sensing Image Semantic Segmentation Sample Generation Using a Decoupled Latent Diffusion Framework

This paper addresses the challenges of sample scarcity and class imbalance in remote sensing image semantic segmentation by proposing a decoupled synthetic sample generation framework based on a latent diffusion model. The method consists of two stages. In the label generation stage, we fine-tune a pretrained latent diffusion model with LoRA to generate semantic label masks from textual descriptions. A novel proportion-aware loss function explicitly penalizes deviations from the desired class distribution in the generated mask. In the image generation stage, we use ControlNet to train a multi-condition image generation network that takes the synthesized mask, along with its text description, as input and produces a realistic remote sensing image. The base Stable Diffusion model’s weights remain frozen during this process, with the trainable ControlNet ensuring that outputs are structurally and semantically aligned with the input labels. This two-stage approach yields coherent image–mask pairs that are well-suited for training segmentation models. Experiments show that models trained on the synthetic samples produced by the proposed method achieve high visual quality and semantic consistency. The proportion-aware loss effectively mitigates the impact of minority classes, boosting segmentation performance on under-represented categories. Results also reveal that adding a suitable proportion of synthetic sample improves segmentation accuracy, whereas an excessive share can cause over-fitting or misclassification. Comparative tests across multiple models confirm the generality and robustness of the approach.