SignSpotter: scene text-informed cross-modal semantic consistency verification for street view images and POI data

Ensuring semantic consistency between street view images and Points of Interest (POI) is critical for reliable geographic entity validation in complex urban scenes. To address challenges such as visual occlusion, scale variation, and ambiguous scene text, this paper proposes SignSpotter, a novel multi-scale vision-language framework for entity-level consistency verification. First, a scene text-informed visual pre-training strategy is introduced to sensitize the Vision Transformer to fine-grained textual regions. Second, a resolution-adaptive asymmetric dual-tower architecture is employed to jointly model multi-scale visual features and POI textual embeddings, effectively mitigating scale disparities. Furthermore, a cross-modal semantic reasoning module establishes precise visual-textual associations via bidirectional attention for explicit consistency inference. Experiments on a large-scale dataset demonstrate that SignSpotter achieves an F1-score of 90.96% on the source city dataset. Cross-city and cross-platform transfer tests further yield stable F1-scores exceeding 86%. These results, together with occlusion analyses, confirm the robustness of SignSpotter and highlight its potential for automated large-scale geographic data quality control.