Subjective Assessment of a Built Environment by ChatGPT, Gemini and Grok: Comparison with Architecture, Engineering and Construction Expert Perception

The emergence of Multimodal Large Language Models (MLLMs) has made methods of artificial intelligence accessible to the general public in a conversational way. It offers tools for the automated visual assessment of the quality of a built environment for professionals of urban planning without requiring specific technical knowledge on computing. We investigated the capability of MLLMs to perceive urban environments based on images and textual prompts. We compared the outputs of several popular models—ChatGPT, Gemini and Grok—to the visual assessment of experts in Architecture, Engineering and Construction (AEC) in the context of a real estate construction project. Our analysis was based on subjective attributes proposed to characterize various aspects of a built environment. Four urban identities served as case studies, set in a virtual environment designed using professional 3D models. We found that there can be an alignment between human and AI evaluation on some aspects such as space and scale and architectural style, and more general accordance in environments with vegetation. However, there were noticeable differences in response patterns between the AIs and AEC experts, particularly concerning subjective aspects such as the general emotional resonance of specific urban identities. It raises questions regarding the hallucinations of generative AI where the AI invents information and behaves creatively but its outputs are not accurate.