Labels, Thresholds and the Reference Itself: Evaluating Sentinel-1 Flood Segmentation Against Copernicus EMS Maps, with a Preregistered Blind Test

Flood-segmentation models for Sentinel-1 synthetic aperture radar (SAR) are usually ranked by cross-validation on benchmark chips whose labels share sensors, dates and definitions with the model inputs. We measure how far such scores carry to operational reference maps, what limits them there, and whether adapting a model to the reference helps. The reference is Copernicus Emergency Management Service (CEMS) rapid mapping: the delineations of 38 flood activations from 2023 to 2026 drawn on non-Sentinel radar satellites, against which every model is applied, without training on these events, at its own calibrated threshold. A gated cross-attention SAR–optical fusion model that scores 0.608 IoU in leave-one-event-out cross-validation on Sen1Floods11 scores 0.266 on these maps, below a physics baseline (0.373); trained on Kuro Siwo's flood labels, the same architecture scores 0.407. Our v1.2 model adds a seasonal Sentinel-1 reference and is trained on 7,006 chips from 138 floods (Kuro Siwo and GEOID-Flood). It scores 0.417 IoU, against 0.405 for TerraMind-base adapted with LoRA to the same labels and tuning budget, and 0.419 against 0.321 for Copernicus's Global Flood Monitoring (GFM) ensemble on the 34 activations GFM covers, with GFM's exclusion mask scored as dry. At each activation's best threshold in hindsight, our model would reach 0.468. The reference is itself uncertain: consecutive CEMS delineations of the same area, made 6–24 h apart, agree with each other at 0.520 macro IoU over 34 activations; on the 27 activations in both sets, the model scores 0.433 and the consecutive products 0.536. Adapting to the reference does not close the remaining gap. In grouped cross-validation over the 38 activations, a threshold refitted on the other folds gains 0.002, and fine-tuning on their CEMS labels loses 0.041 (0.380 against 0.421), so an adoption rule fixed in advance adopted neither. Because this benchmark had been seen during development, all of these comparisons are exploratory. The confirmatory test is preregistered: every eligible CEMS flood activation from EMSR899 on is scored by models frozen at a git tag, and the hashes of the sealed scores are committed to a hash-chained ledger until 20 activations in 15 clusters are scored, or until the end of 2027. The frozen weights are public and drive a service that maps new floods from Sentinel-1, showing the model's map with no reference and no score. We also report two errata in our earlier results.

Authors

Publication Details

Published
2026-11-14
DOI
https://doi.org/10.31223/x5fz4m
Primary Topic
Flood Risk Assessment and Management
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Labels, Thresholds and the Reference Itself: Evaluating Sentinel-1 Flood Segmentation Against Copernicus EMS Maps, with a Preregistered Blind Test

Teo Clerici Jurado
Flood Risk Assessment and Management
preprint

Labels, Thresholds and the Reference Itself: Evaluating Sentinel-1 Flood Segmentation Against Copernicus EMS Maps, with a Preregistered Blind Test

Teo Clerici Jurado
preprint en

Abstract

Flood-segmentation models for Sentinel-1 synthetic aperture radar (SAR) are usually ranked by cross-validation on benchmark chips whose labels share sensors, dates and definitions with the model inputs. We measure how far such scores carry to operational reference maps, what limits them there, and whether adapting a model to the reference helps. The reference is Copernicus Emergency Management Service (CEMS) rapid mapping: the delineations of 38 flood activations from 2023 to 2026 drawn on non-Sentinel radar satellites, against which every model is applied, without training on these events, at its own calibrated threshold. A gated cross-attention SAR–optical fusion model that scores 0.608 IoU in leave-one-event-out cross-validation on Sen1Floods11 scores 0.266 on these maps, below a physics baseline (0.373); trained on Kuro Siwo's flood labels, the same architecture scores 0.407. Our v1.2 model adds a seasonal Sentinel-1 reference and is trained on 7,006 chips from 138 floods (Kuro Siwo and GEOID-Flood). It scores 0.417 IoU, against 0.405 for TerraMind-base adapted with LoRA to the same labels and tuning budget, and 0.419 against 0.321 for Copernicus's Global Flood Monitoring (GFM) ensemble on the 34 activations GFM covers, with GFM's exclusion mask scored as dry. At each activation's best threshold in hindsight, our model would reach 0.468. The reference is itself uncertain: consecutive CEMS delineations of the same area, made 6–24 h apart, agree with each other at 0.520 macro IoU over 34 activations; on the 27 activations in both sets, the model scores 0.433 and the consecutive products 0.536. Adapting to the reference does not close the remaining gap. In grouped cross-validation over the 38 activations, a threshold refitted on the other folds gains 0.002, and fine-tuning on their CEMS labels loses 0.041 (0.380 against 0.421), so an adoption rule fixed in advance adopted neither. Because this benchmark had been seen during development, all of these comparisons are exploratory. The confirmatory test is preregistered: every eligible CEMS flood activation from EMSR899 on is scored by models frozen at a git tag, and the hashes of the sealed scores are committed to a hash-chained ledger until 20 activations in 15 clusters are scored, or until the end of 2027. The frozen weights are public and drive a service that maps new floods from Sentinel-1, showing the model's map with no reference and no score. We also report two errata in our earlier results.

Flood Risk Assessment and Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon