A framework for the evaluation of flood inundation predictions over extensive benchmark databases

Accurate Flood Inundation Mapping (FIM) is essential for forecasting and evaluation. Traditional pixel-based approaches can be time-intensive and error-prone. Here, we introduced the Flood Inundation Mapping Evaluation Framework (FIMeval), an open-source toolset for large-scale FIM evaluation. FIMeval links to a benchmarking database that includes high-quality FIM benchmarks across the Contiguous United States, derived from remote sensing and high-fidelity model-predicted datasets. FIMeval supports pixel-based metrics and integrates impact-based assessments using building footprint data. We demonstrated its application using (a) high-resolution aerial imagery FIM for 2016 Midwest Flood (b) remote sensing-derived benchmarks from Hurricane Matthew (2016), and (b) simulated 100-year and 500-year FIM across 45 Hydrologic Unit Code-8 watersheds using the Federal Emergency Management Agency's Base Level Engineering dataset. The NOAA Office of Water Prediction Height Above Nearest Drainage (OWP HAND-FIM) was the model-predicted FIM for all case studies. We tested the influence of data-imbalance on the scores using two inbuilt methods. • FIMeval is an open-source framework for large-scale flood inundation mapping evaluation. • FIMeval supports pixel-based metrics and impact-based assessments using building footprint data. • FIMeval can be executed on cloud platforms like 2i2c and Google Colab.