BFTNet: Boundary-Induced Four-Phase Transformer Network for RGB-Thermal Urban Road Scene Parsing

The use of boundary information in scene parsing has attracted increasing research interest. In some scenes, such as those under poor lighting conditions, chaotic appearance, and occlusion, the integrity and accuracy of scene parsing are drastically degraded. To address these problems, we introduce a boundary-induced four-phase transformer network (BFTNet) for urban road scene parsing of color pairs and thermal images. In the first phase, we use a novel boundary optimization module to leverage rich semantic information and boundary details to integrate multiscale local and global features captured and weighted by a backbone network and channel attention. Then, we use the optimized boundary features in a cross-scale feature fusion module to further improve the scene parsing accuracy in the second phase. Finally, a cross-scale feature fusion module is adopted aiming at integrating convolutional neural network features in the foreground region. Comprehensive experimental results show that the proposed BFTNet outperforms 14 state-of-the-art approaches in terms of various indicators on two scene parsing datasets.