Speeding-up diffusion models for remote sensing semantic segmentation

Denoising Diffusion Probabilistic Models (DDPMs) have demonstrated exceptional potential across various generative modeling tasks. Despite evident promise in semantic segmentation, their adoption for remote sensing remains limited primarily due to computationally demanding inference. While initial approaches using DDPMs in remote sensing achieve competitive accuracy with state-of-the-art models, the multi-step nature of their image generation process poses a major bottleneck. To address this limitation, this paper investigates three key strategies for accelerating inference: (1) optimizing training and inference steps, (2) applying DDPM acceleration techniques adapted to segmentation task (including Denoising Diffusion Implicit Models, Improved Denoising Diffusion Models, and Progressive Distillation), and (3) thoroughly analyzing the trade-off between accuracy improvement and additional inference time when using test-time augmentation. These strategies are extensively tested with two established remote sensing semantic segmentation datasets focused on buildings and roads. Finally, we compare the optimized diffusion-based model with state-of-the-art convolutional-based models in terms of accuracy and inference times, showing the narrowing gap between both approaches and the increasing viability of diffusion-based segmentation for practical applications. • Optimized DDPMs with fewer steps for speeding up remote sensing segmentation. • Progressive Distillation beats other DDPM acceleration methods in segmentation. • Test-time augmentations benefit more Diffusion models than convolutional models. • Accelerated DDPM achieves SOTA building segmentation results 32x faster. • Road segmentation with DDPMs is challenging and requires future research.