A Benchmark Hit Is Not an Exposure: What Published Vision-Language Geolocation Accuracy Says About Location Privacy, Why One Model Locates 61 Percent of One Photo Set to a Kilometre and Fewer Than 12 Percent of Another to a Hundred, and the Exposure

(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research Since 2024 a growing literature has argued that general-purpose vision-language models can work out where a photograph was taken well enough to make image geolocation an immediate privacy risk, and a second literature has begun building defences against it. In this analysis the author reviews and synthesises published results to show that the direction of that claim survives the evidence but its usual support does not: the accuracies quoted for it are properties of particular photo sets, adversaries and distance thresholds, and none of them is the quantity a privacy claim is about. The same model places 61 percent of a curated, text-rich stock-photo set within one kilometre, about 23 percent of a public Flickr benchmark, and fewer than 12 percent of sampled street-level images within one hundred kilometres. One video-frame benchmark reports 65.6 percent within a kilometre for clips from one platform and 6.4 percent for clips from another. The obvious deflation, that benchmark images were seen in training, does not explain the capability: on photographs collected by the authors and not drawn from any public source, frontier models still name the city about 60 percent of the time, and freshly taken photographs of Seoul are located about as well as Street View images of the same city. The author separates five variables that published results leave unfixed (image population, harm radius, adversary configuration, counterfactual baseline, and the unit of exposure), shows that defences are mostly scored against a fixed adversary although a published photograph must resist every later one, and proposes an exposure protocol that states each variable before a number is reported.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-20
DOI
https://doi.org/10.5281/zenodo.24382474
Primary Topic
Privacy-Preserving Technologies in Data
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

A Benchmark Hit Is Not an Exposure: What Published Vision-Language Geolocation Accuracy Says About Location Privacy, Why One Model Locates 61 Percent of One Photo Set to a Kilometre and Fewer Than 12 Percent of Another to a Hundred, and the Exposure

Pranay M. Mahendrakar
Zenodo (CERN European Organization for Nuclear Research)
Privacy-Preserving Technologies in Data
preprint

A Benchmark Hit Is Not an Exposure: What Published Vision-Language Geolocation Accuracy Says About Location Privacy, Why One Model Locates 61 Percent of One Photo Set to a Kilometre and Fewer Than 12 Percent of Another to a Hundred, and the Exposure

Pranay M. Mahendrakar
preprint en

Abstract

(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research Since 2024 a growing literature has argued that general-purpose vision-language models can work out where a photograph was taken well enough to make image geolocation an immediate privacy risk, and a second literature has begun building defences against it. In this analysis the author reviews and synthesises published results to show that the direction of that claim survives the evidence but its usual support does not: the accuracies quoted for it are properties of particular photo sets, adversaries and distance thresholds, and none of them is the quantity a privacy claim is about. The same model places 61 percent of a curated, text-rich stock-photo set within one kilometre, about 23 percent of a public Flickr benchmark, and fewer than 12 percent of sampled street-level images within one hundred kilometres. One video-frame benchmark reports 65.6 percent within a kilometre for clips from one platform and 6.4 percent for clips from another. The obvious deflation, that benchmark images were seen in training, does not explain the capability: on photographs collected by the authors and not drawn from any public source, frontier models still name the city about 60 percent of the time, and freshly taken photographs of Seoul are located about as well as Street View images of the same city. The author separates five variables that published results leave unfixed (image population, harm radius, adversary configuration, counterfactual baseline, and the unit of exposure), shows that defences are mostly scored against a fixed adversary although a published photograph must resist every later one, and proposes an exposure protocol that states each variable before a number is reported.

Zenodo (CERN European Organization for Nuclear Research)
Privacy-Preserving Technologies in Data
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon