Urban Perception Extraction from Texts Shared on Social Media: Framework and Applications

In this Ph.D. thesis, we proposed an automatic and generic framework composed of 5 major layers (Data Collection, Preprocessing and Embeddings, Models Training, Knowledge Extraction, and Applications) to extract the user’s urban perception from Location-Based Social Network (LBSN) data. The framework employs advanced deep learning algorithms, including sentence embeddings, to capture lexical and semantic relationships in textual data and, thus, effectively extract content related to urban perceptions from LBSNs. Moreover, this framework circumvents the need for labor-intensive field surveys or manual extraction processes, enabling scalable and real-time analysis of urban perception. Studying some urban areas from Chicago, New York City, and London, we demonstrate the framework’s effectiveness in extracting valuable insights related to urban perceptions from LBSN data. Furthermore, we conducted a comparative evaluation using a public dataset derived from volunteers’ perceptions in a controlled experiment, where it was possible to observe that both results yielded a very similar level of agreement. Finally, we introduce a novel tool called Real-Estate Urban Perceptions (REAL-UP), which aims to enhance the real-estate marketplace as a proof-of-concept for our work. REAL-UP provides rich knowledge regarding urban areas in the form of interactive 2D maps, more specifically, the emotion and sentiment perceived, and a short review generated by a Large Language Model (LLM) based on LBSN messages, for every city’s neighborhood, in addition to information commonly provided by such applications, as rent price, property type, and so on.