Omnidirectional semantic segmentation fusion network with cross-stage and cross-dimensional remodeling

Multi-scale networks extract features at different scales through multiple stages, which have different spatial contexts and semantic information. In response to the challenge of insufficient consideration of multidimensional relationships in existing feature concatenation leading to imbalanced contextual fusion., our paper proposes an omnidirectional semantic segmentation fusion network called OFNet, which is based on Dual-Branch Gradient Perception Enhancement (DBGPE) and Ternary Cross-Dimensional Dependency (TCDD). For multi-scale fusion features extracted from the backbone, Dual-Branch Gradient Perception Enhancement utilizes the Gradient Enhancement Branch (GEB) and Attention Branch‘(AB) to achieve cross-stage contextual perception enhancement processing of features. Subsequently, we employ the Ternary Cross-Dimensional Dependency to perform a double rotation operation on the features and splice them with the input features to establish a connection between the channel dimension and the spatial dimension. The method proposed in our paper involves cross-stage and cross-dimensional remodeling of multi-scale fused features, fully unleashing the expressive power of fused features. We verified the effectiveness of OFNet on three public datasets: Cityscapes, ADE 20K, and COCO-Stuff 164K. OFNet achieved 80.4% mIoU with only 5.76M parameters on Cityscapes, reaching the state-of-the-art in lightweight semantic segmentation. The performance on the ADE 20K and COCO-Stuff 164K datasets is also highly competitive.