LLM-Vision in enhancing the understanding of public spaces

Urban geography traditionally focuses on material structure, populations, and services within a city.However, diversity of architectural styles, street layouts, presence of utilities and variety in uses across time has been a difficult aspect to monitor.It was possible to perform scene classification, objected detection and automated observations of traffic, footfall, and crowd dynamics using custom machine learning models requiring large amount of data or manual review.Additionally, performing these operations using available models without pre-training yielded unacceptably inaccurate results as models were not exposed to samples from an unknown context.This has rendered the task of deploying such solutions impractical for all but those engaged in the most advanced research and development efforts.Recent advancements and public availability of Large Language Model with Vision (OpenAI, 2024) have made the task of zeroshot scene understanding attainable, and we will verify it in this work.