Evaluating urban street views through collective emotional perception from multimodal social media cityscape posts: An R-ECoT-based interpretable framework

Urban street views shape everyday urban experience, yet their assessment still relies largely on researcher-defined indicators and structured elicitation, with limited use of naturally occurring public responses. This study develops an interpretable framework that transforms social-media image–comment interactions into eight-dimensional collective emotional appraisals of urban scenes. Under approved access, 1078 Facebook and Instagram cityscape posts and 20,546 comments were retrieved through the SOMAR Virtual Data Enclave. Comment-derived Plutchik labels were assessed through human validation and robustness tests, then used as weak supervision for R-ECoT, which jointly predicts emotion intensities and generates emotion-specific grounding maps from images. The trained CLIP-based model was applied to 610,017 ZENRIN street-view images in Setagaya Ward, Tokyo, and predictions were aggregated to 10 m road segments. Comment-derived labels agreed more closely with an independent post-level human reference than the strongest image-only estimate (Pearson 0.818 versus 0.751). CLIP-R-ECoT achieved the best performance in hierarchical benchmarks, with a KL divergence of 0.329, an MSE of 0.0106, and a balanced accuracy of 0.757. Its grounding maps showed greater random-corrected masking faithfulness than Grad-CAM (0.083 versus 0.049), higher perturbation stability, and lower spatial fragmentation. Street-level mapping revealed differentiated emotional patterns, while emotion-specific high-response regions in the images were associated with distinct semantic, color-atmospheric, and depth-structural attributes. By converting dispersed public reactions into spatially continuous and interpretable evidence, the framework supports fine-grained streetscape screening, helps identify locations warranting closer inspection, and directs design attention toward the visible conditions associated with different public responses.