LLM-Assisted Detection and Community-Based Review of Promotional Texts in OpenStreetMap: An Empirical Study
Promotional and SEO-oriented free-text content can undermine the neutrality and quality of OpenStreetMap (OSM). Because such text is multilingual and context-dependent, purely rule-based detection is challenging. This study evaluates large language models (LLMs) for identifying promotional content in OSM description=* tags and investigates their use in a community-based quality assurance workflow. A classification prompt was developed using a pilot dataset and evaluated on a balanced, manually annotated multilingual dataset of 1500 descriptions. Thirty-five open-weight model configurations were compared using precision, recall, F1-score, and runtime, and two conventional text-classification baselines were evaluated for classification performance. The best-performing model, llama3.3:70b, achieved 95.03% precision, 91.87% recall, and an F1-score of 93.42%, outperforming the evaluated conventional baselines. This model was then applied to 16,156 newly added or modified OSM descriptions in a real-world workflow. It identified 2738 descriptions as potentially promotional, resulting in 2599 public changeset comments. Within a 28-day follow-up period, 79.5% of analyzed commented cases were modified or removed, compared with 27.3% during a later comparison period without workflow-generated comments. Because the two conditions were temporally separated and non-randomized, this difference should be interpreted as an observational association.
Authors
- Pascal Neis (ORCID: https://orcid.org/0000-0002-5158-796X)
Institutions
- University of Applied Sciences Mainz (DE)
Publication Details
- Journal
- ISPRS International Journal of Geo-Information
- Published
- 2026-11-18
- DOI
- https://doi.org/10.3390/ijgi15110514
- Primary Topic
- Geographic Information Systems Studies
- Type
- article
- Field-Weighted Citation Impact
- 0.00