Simulation-based validation of a method to detect changes in SARS-CoV-2 reinfection risk

Given the high global seroprevalence of SARS-CoV-2, understanding the risk of reinfection has become increasingly important. Models developed to track trends in reinfection risk should be robust against possible biases arising from imperfect data observation processes. We performed simulation-based validation of an existing catalytic model designed to detect changes in the risk of reinfection by SARS-CoV-2. The catalytic model assumes the risk of reinfection is proportional to observed infections. Validation involved using simulated primary infections, consistent with the number of observed infections in South Africa. To assess the performance of the catalytic model, we simulated reinfection datasets that incorporated different processes that may bias inference, including imperfect observation and mortality. A Bayesian approach was used to fit the model to simulated data, assuming a negative binomial distribution around the expected number of reinfections, and model projections were compared to the simulated data using different magnitudes of change in reinfection risk. We assessed the model's ability to accurately detect changes in reinfection risk when included in the simulations, as well as the occurrence of false positives when reinfection risk remained constant. The model parameters converged in most scenarios leading to model outputs aligning with anticipated outcomes. The model successfully detected changes in the risk of reinfection when such a change was introduced to the data. Low observation probabilities (10%) of both primary- and reinfections resulted in low numbers of observed cases from the simulated data and poor convergence. The model's performance was assessed on simulated data representative of the South African SARS-CoV-2 epidemic, reflecting its timing of waves and outbreak magnitude. Model performance under similar scenarios may be different in settings with smaller epidemics (and therefore smaller numbers of reinfections). Ensuring model parameter convergence is essential to avoid false-positive detection of shifts in reinfection risk. While the model is robust in most scenarios of imperfect observation and mortality, further simulation-based validation for regions experiencing smaller outbreaks is recommended. Caution must be exercised in directly extrapolating results across different epidemiological contexts without additional validation efforts.