Beyond Scarce Labels: Foundation Models for Mapping Oil, Gas, and Power Infrastructure

Foundation models recognize objects in overhead imagery, but operational Earth observation must also discover unfamiliar assets, separate facilities from equipment, localize them across scales, and explain each conclusion. Industry-specific geospatial datasets are scarce: energy taxonomies are specialized, labels are geographically uneven, and expert annotation is slow and costly. We present a training-free workflow with a provider-agnostic interface that organizes pretrained vision-language, open-vocabulary detection, and segmentation models as a structured geospatial observer. A scene model proposes candidate asset types, counts, and expected patterns. A bounded evidence policy decides where to look more closely, and a typed evidence graph consolidates proposals, preserves facility-equipment relations, arbitrates duplicates, and gates local geometry for dense or repeated structures. Each output is a record with confidence, provenance, processing cost, and explicit uncertainty. Evaluation uses frozen protocols, paired controls, and predeclared claim boundaries. On 300 external aerial images of wind infrastructure, processed without any benchmark taxonomy in the prompts, target-discovery F1 is 0.966 (95% CI 0.943–0.985) versus 0.816 for a fixed-vocabulary open-vocabulary detector, while box localization is weaker (F1 0.209 versus 0.316). On 300 label-blind French aerial images, it raises 11 false alarms on 167 rooftops without solar panels versus 111 for OWLv2, at lower recall (0.52 versus 0.95). On 60 site-disjoint energy sites (well pads, pumpjacks, refineries, terminals, compressor and processing sites, substations, power plants), processed with the benchmark taxonomy declared, entity-center F1 (0.553) far exceeds strict box F1 (0.255): assets are often found but their extent is mis-drawn. The full workflow improves strict F1 by 0.034 over flat fusion of the same primary proposals, and graph-gated local geometry recovers dense repeated objects on DIOR (+0.037 strict F1). Rare-energy scene classification does not outperform a matched vision-language baseline. The workflow can shift scarce expert effort from exhaustive annotation toward verification. Facility extent and open-taxonomy discovery remain the main limitations.

Authors

Institutions

Publication Details

Published
2026-11-12
DOI
https://doi.org/10.22541/essoar.15012143/v1
Primary Topic
Advanced Neural Network Applications
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Beyond Scarce Labels: Foundation Models for Mapping Oil, Gas, and Power Infrastructure

Jade Eva Guisiano, Ziheng Zhu, Da Zhang
Advanced Neural Network Applications
preprint

Beyond Scarce Labels: Foundation Models for Mapping Oil, Gas, and Power Infrastructure

Jade Eva Guisiano, Ziheng Zhu, Da Zhang
preprint en

Abstract

Foundation models recognize objects in overhead imagery, but operational Earth observation must also discover unfamiliar assets, separate facilities from equipment, localize them across scales, and explain each conclusion. Industry-specific geospatial datasets are scarce: energy taxonomies are specialized, labels are geographically uneven, and expert annotation is slow and costly. We present a training-free workflow with a provider-agnostic interface that organizes pretrained vision-language, open-vocabulary detection, and segmentation models as a structured geospatial observer. A scene model proposes candidate asset types, counts, and expected patterns. A bounded evidence policy decides where to look more closely, and a typed evidence graph consolidates proposals, preserves facility-equipment relations, arbitrates duplicates, and gates local geometry for dense or repeated structures. Each output is a record with confidence, provenance, processing cost, and explicit uncertainty. Evaluation uses frozen protocols, paired controls, and predeclared claim boundaries. On 300 external aerial images of wind infrastructure, processed without any benchmark taxonomy in the prompts, target-discovery F1 is 0.966 (95% CI 0.943–0.985) versus 0.816 for a fixed-vocabulary open-vocabulary detector, while box localization is weaker (F1 0.209 versus 0.316). On 300 label-blind French aerial images, it raises 11 false alarms on 167 rooftops without solar panels versus 111 for OWLv2, at lower recall (0.52 versus 0.95). On 60 site-disjoint energy sites (well pads, pumpjacks, refineries, terminals, compressor and processing sites, substations, power plants), processed with the benchmark taxonomy declared, entity-center F1 (0.553) far exceeds strict box F1 (0.255): assets are often found but their extent is mis-drawn. The full workflow improves strict F1 by 0.034 over flat fusion of the same primary proposals, and graph-gated local geometry recovers dense repeated objects on DIOR (+0.037 strict F1). Rare-energy scene classification does not outperform a matched vision-language baseline. The workflow can shift scarce expert effort from exhaustive annotation toward verification. Facility extent and open-taxonomy discovery remain the main limitations.

Fudan University (CN), Tsinghua University (CN)
Advanced Neural Network Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon