Integrating Multiple Semantics of Street View Imagery for Semi-Supervised Building Function Identification

Building function identification plays a crucial role in providing basic data for urban planning, management, and various intelligent applications. Today, building function identification methods using Street View Images (SVIs) have made significant progress. However, these methods use the visual features of SVIs to infer building functions, which ignores the contributions of the multiple potential semantics of SVIs, resulting in suboptimal identification accuracy. To address this issue, this study proposes a multi-semantic semi-supervised building function identification (MS-SS-BFI) method, which integrates multi-level visual semantics and spatial contextual semantics to improve building function identification from SVIs. Specifically, a location mapping module was designed to align SVIs with buildings. Additionally, a multi-level visual semantic extraction module was developed to integrate the visual semantics and visual-textual semantics of SVIs. In addition, a semi-supervised spatial interaction module was designed to characterize the spatial context of buildings. Extensive experiments on the Brooklyn dataset show that the proposed method achieves 7.98% improvement in F1-score over the state-of-the-art baseline, demonstrating superior performance and robustness. This work explores a novel approach to building function identification and provides a methodological reference for various SVI-based applications.