Participation bias and false alarms in resistant-carriage surveillance: a simulation study

Abstract Background Changes in resistant carriage among surveillance participants can reflect population change or changing participation. We examined whether explicit participation modelling and threshold calibration control false alarms, and whether good model fit identifies the population change. Methods Two mechanistic carriage structures generated synthetic surveillance at an exposed and a control site. The target was the population resistant-carriage difference from an unshocked counterfactual, averaged over 180 days. Joint sensitive/resistant measurements included participation selection, clustered sampling and diagnostic error. A finite library supported early-survey rules with and without temporal projection. Calibration, selection and evaluation used 17, 29 and 44 scenarios; evaluation used 3,000 replicates per scenario. Follow-ups examined participation coverage, a calibrated direct comparator, and 132,000 synthetic library fits at sensitive-carrier participation weights 0.8, 1.0 and 1.2 without recalibration. A deterministic diagnostic tested whether early observations identified perturbation duration. Results Participation shifts alone produced apparent resistant-carriage changes of approximately − 4.0 to + 3.7 percentage points. Under selection on resistant carriage alone, widening the library while calibrating against all original nulls reduced the dynamic rule's maximum false-alarm rate from 6.63% to 3.03% and increased mean power from 11.47% to 19.24%. At the widest library, mean power was 25.42% without projection and 11.02% for the calibrated direct comparator; corresponding maximum false-alarm rates were 2.97% and 2.77%. In the new stress test, sensitive-carrier weight 1.2 raised maximum false-alarm rates to 77.2% and 76.9% for the two model rules. Across all 44 scenarios at that weight, 99.93% of fits had small observed-summary residual distance, yet 63.13% combined small distance with population-target error exceeding 1 percentage point. Conclusions Modelling participation improved detection within the assumed selection process, but an unrepresented participation change invalidated the calibrated model rules despite close fit to observations. Low power also limits reassurance from a non-alert. Resistant-carriage monitoring requires explicit assessment of participation assumptions and external validation before these rules can guide infection-control decisions.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-16
DOI
https://doi.org/10.21203/rs.3.rs-11295092/v1
Primary Topic
Data-Driven Disease Surveillance
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Participation bias and false alarms in resistant-carriage surveillance: a simulation study

Chaokun Hong
2 links
Research Square
Data-Driven Disease Surveillance
preprint

Participation bias and false alarms in resistant-carriage surveillance: a simulation study

Chaokun Hong
preprint en

Abstract

Abstract Background Changes in resistant carriage among surveillance participants can reflect population change or changing participation. We examined whether explicit participation modelling and threshold calibration control false alarms, and whether good model fit identifies the population change. Methods Two mechanistic carriage structures generated synthetic surveillance at an exposed and a control site. The target was the population resistant-carriage difference from an unshocked counterfactual, averaged over 180 days. Joint sensitive/resistant measurements included participation selection, clustered sampling and diagnostic error. A finite library supported early-survey rules with and without temporal projection. Calibration, selection and evaluation used 17, 29 and 44 scenarios; evaluation used 3,000 replicates per scenario. Follow-ups examined participation coverage, a calibrated direct comparator, and 132,000 synthetic library fits at sensitive-carrier participation weights 0.8, 1.0 and 1.2 without recalibration. A deterministic diagnostic tested whether early observations identified perturbation duration. Results Participation shifts alone produced apparent resistant-carriage changes of approximately − 4.0 to + 3.7 percentage points. Under selection on resistant carriage alone, widening the library while calibrating against all original nulls reduced the dynamic rule's maximum false-alarm rate from 6.63% to 3.03% and increased mean power from 11.47% to 19.24%. At the widest library, mean power was 25.42% without projection and 11.02% for the calibrated direct comparator; corresponding maximum false-alarm rates were 2.97% and 2.77%. In the new stress test, sensitive-carrier weight 1.2 raised maximum false-alarm rates to 77.2% and 76.9% for the two model rules. Across all 44 scenarios at that weight, 99.93% of fits had small observed-summary residual distance, yet 63.13% combined small distance with population-target error exceeding 1 percentage point. Conclusions Modelling participation improved detection within the assumed selection process, but an unrepresented participation change invalidated the calibrated model rules despite close fit to observations. Low power also limits reassurance from a non-alert. Resistant-carriage monitoring requires explicit assessment of participation assumptions and external validation before these rules can guide infection-control decisions.

Research Square
University of Oxford (GB)
United Nations High Commissioner for Refugees, European Centre for Medium-Range Weather Forecasts
Good health and well-being
Data-Driven Disease Surveillance
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon