Urban Historical Building Style Recognition Method Integrating Street-view Imagery and Historical Text Semantics

To address the problems that single visual models in urban historical building style recognition are vulnerable to occlusion, facade renovation and style mixing, and that they have difficulty representing historical semantics, this paper proposes HVTS-Net, a building style recognition method that integrates street-view visual features and historical text semantics. The method takes building instance images, building masks, structured visual features, historical texts and style semantic prototypes as inputs, and realizes automatic recognition and interpretable analysis of historical building styles through visual encoding, text encoding, cross-modal semantic-guided fusion and semantic prototype constraints. A multimodal dataset containing 846 street-view sampling points, 2,946 valid street-view images, 4,612 building instance samples and 1,368 historical text records was constructed, covering six categories of historical building styles. The results show that the Accuracy, Precision, Recall, Macro-F1 and Top-3 Accuracy of HVTS-Net reach 91.38%, 90.87%, 90.26%, 90.56% and 98.12%, respectively, with improvements of 4.46 and 4.88 percentage points over the model without the historical text branch. The ablation experiment shows that Macro-F1 decreases to 85.68% after removing the historical text branch and to 87.24% after removing the cross-modal fusion module. Under enhanced shop-sign occlusion, the model still maintains a Macro-F1 of 84.86%. Spatial analysis shows that Minnan red-brick residential style, arcade commercial building style and modern renovated mixed style account for 22.46%, 19.82% and 21.42%, respectively. The proposed method can provide technical support for historical district style identification, conservation zoning and renewal management.