Construction of a multiscale feature fusion model for indoor scene recognition and semantic segmentation

Fast and accurate extraction of multi-scale semantic information is crucial for indoor applications such as navigation and personalized services. This paper focuses on large-scale indoor scene categories and small-scale indoor element semantics, proposing an RGB-based dual-task model for scene recognition and semantic segmentation. The model employs a pyramid pooling module to create a shared feature layer for both tasks. The scene recognition branch incorporates an SE attention mechanism and a Transformer module to enhance scene understanding, while the semantic segmentation branch fuses low-level features from ResNet50 to improve learning of shapes and textures. This paper investigates the fusion methods of scene-level and element-level features and explores the optimal training strategy for the dual-task model. It also discusses and analyzes the impact of different weight combinations of dual-task Loss values on the model's performance. The experimental results demonstrate that the overall accuracy (OA) of scene recognition and semantic segmentation of this method is 98.4%, and the overall accuracy (OA) of semantic segmentation is 82.6%. The average time for processing a single scene is approximately 0.037 s, which is superior to the latest models compared in this paper. It is found that the optimal training strategy for a single task is conducive to enhancing the accuracy of dual tasks. Moreover, when the loss weights of the two tasks in this method are both 1, the comprehensive effect of the model is the best.