Spatial Plaid Attention Decoder for Semantic Segmentation

Striking a balance between efficiency and accuracy is a challenge in the design and implementation of decoders. Accurate decoders often tend to be highly complex and computationally costly. This paper presents a novel decoder for semantic segmentation: Spatial Plaid Attention Decoder (SPADe). We propose a Spatial Plaid Attention module that performs an efficient local feature collection through spatial feature folding to help the model capture the local structure of objects while using long-range feature aggregation to consider the global structures efficiently and accurately. This makes SPADe a suitable choice for applications with limited resources. With a size of 7.3% UPerNet decoder, our SPADe obtains state-of-the-art performance with several popular backbones on public benchmarks. On Cityscapes and ADE20K, SPADe obtains 84.3% and $53.8 \% \mathrm{mIoU}$, while reducing the total GFlops by 32.8% and 70.5%, respectively. We also demonstrate that the effective design of SPADe allows it to capture long-range dependencies with a large receptive field. Implementation of the SPADe is available at github/SPADe.