Applying and adapting established frameworks for geospatial data quality to evaluate Generative AI in geographic information tasks

Challenges such as hallucination and non-determinism are well known to undermine the reliability of Generative AI (GenAI) applications. Despite this, their use across multiple fields and domains continues to rapidly grow. Many GenAI applications now generate or interact with geospatial information – leading to the potential proliferation of non-human generated geospatial data. Because such tools can be developed without specialist geographic expertise, there is a growing risk of diminished oversight and declining geospatial data quality. This paper presents an approach for evaluating the geospatial information produced by large language models (LLMs) using established frameworks for geospatial data quality. We apply this framework in two experiments that assess the completeness, accuracy, precision, and consistency of LLM outputs: geoparsing and route finding. Interpreting these results through ontological and teleological perspectives reveals fundamental limitations in how LLMs represent and reason about space. The study highlights the need for responsible evaluation and adaptation of data quality standards as GenAI becomes increasingly embedded within geographic information science.