Transformer-Driven Multimodal Emotion and Interaction Intent Understanding and Analysis for Urban Design

In the field of urban design, accurately and efficiently understanding the complex emotions and interaction intentions conveyed by citizens through multimodal feedback (text, images, speech) is a key challenge. To address this challenge and the shortcomings of traditional survey methods and existing general analysis tools, this paper proposes an intelligent analysis framework called UrbanMultimodalInsight (UMI). At the core of UMI is its innovatively designed Hierarchical Cross-Modal Attention Fusion Transformer (H-CRAFT) model. Through refined unimodal feature adaptation, deep pairwise cross-modal information interaction enhancement, context-aware dynamic gating, and final deep fusion, this model can effectively integrate heterogeneous data to jointly identify citizen emotions and core intentions. Its analysis results can be applied to assist design decisions through appropriate presentation methods. To rigorously evaluate the model’s performance, this study constructed a dedicated urban design multimodal emotion and intent dataset (UrbanDesign-MMIntent). Comprehensive experimental validation on this dataset shows that the UMI (H-CRAFT) framework exhibits excellent performance: its exact match rate (EMR) on the joint emotion and intent prediction task reached 0.657, achieving a significant performance improvement of approximately 7.35% compared to the current best-performing baseline model (EMR of 0.612). This research not only validates the effectiveness of the proposed method but also provides key technical support and valuable empirical evidence for enhancing the human-centeredness and intelligence of urban design.