ETFE: Efficient transformer with feature enhancement for semantic segmentation

Abstract Despite the powerful capabilities of Transformers in relation to modeling, their effectiveness in capturing multi-scale object features in image-based tasks is often limited. This limitation arises from the diverse shapes and distances between objects in natural scenes, which pose challenges for Transformers to accurately capture object sizes and distances. To overcome this problem, we propose an effective Transformer which extracts multi-scale features at different resolutions. First, we leverage a multi-head attention mechanism to extract richer semantic features at various resolutions. The semantic features are then passed through the Feature Gate to aggregate multi-scale semantic features. Finally, Attentional Feature Enhancement further strengthens the semantic information of target features, enabling the modeling of dependencies among multi-scale features. Comprehensive experiments conducted on the Cityscapes and ADE20K datasets have shown that our model outperforms competing methods in both performance and accuracy.