Calibration and Configurational Recovery in NCA and fsQCA: A Monte Carlo Benchmark for Public-Service Research in Vietnam

Abstract Calibration and scope choices determine what set-theoretic estimands represent, yet applied studies often treat them as neutral preprocessing. We benchmark the recovery and false-detection behaviour of Necessary Condition Analysis (NCA) and fuzzy-set Qualitative Comparative Analysis (fsQCA) using a public-service measurement framework motivated by five rural services in Vietnam. The evaluation uses 456,000 synthetic datasets from 456 scenarios with 1,000 replications each. Scenarios vary sample size, measurement error, condition correlation, cellwise missingness, outcome resolution, missingness mechanism and two data-generating processes: a two-condition gate and a substitutive non-gate. Across the 432-scenario missing-completely-at-random factorial, fsQCA calibrated with substantive 2/3/4 anchors recovered the programmed gate in 0.749 of replications and falsely detected the pair in 0.015 of non-gate replications. Raw-score NCA yielded rates of 0.673 and 0.087, respectively. NCA applied to 2/3/4 fuzzy memberships was conservative (0.049 recovery), whereas percentile-calibrated fsQCA was less specific (0.519 recovery; 0.162 false detection). In matched scenarios, increasing sample size improved semantic-anchor fsQCA recovery but reduced raw NCA recovery as observations filled the ceiling region. These results show that calibration anchors and NCA scope definitions are part of the estimand. Applied studies should preregister external anchors, report scope and algorithmic non-results, and test recovery against a plausible counterfactual data-generating process. The benchmark evaluates analytical workflows; it does not estimate how Vietnamese residents assess public services.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-16
DOI
https://doi.org/10.21203/rs.3.rs-11118533/v1
Primary Topic
Qualitative Comparative Analysis Research
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Calibration and Configurational Recovery in NCA and fsQCA: A Monte Carlo Benchmark for Public-Service Research in Vietnam

Trần Tuấn Anh, Nguyen Thanh Vu, Thi Thu Hoai Pham
2 links
Research Square
Qualitative Comparative Analysis Research
preprint

Calibration and Configurational Recovery in NCA and fsQCA: A Monte Carlo Benchmark for Public-Service Research in Vietnam

Trần Tuấn Anh, Nguyen Thanh Vu, Thi Thu Hoai Pham
preprint en

Abstract

Abstract Calibration and scope choices determine what set-theoretic estimands represent, yet applied studies often treat them as neutral preprocessing. We benchmark the recovery and false-detection behaviour of Necessary Condition Analysis (NCA) and fuzzy-set Qualitative Comparative Analysis (fsQCA) using a public-service measurement framework motivated by five rural services in Vietnam. The evaluation uses 456,000 synthetic datasets from 456 scenarios with 1,000 replications each. Scenarios vary sample size, measurement error, condition correlation, cellwise missingness, outcome resolution, missingness mechanism and two data-generating processes: a two-condition gate and a substitutive non-gate. Across the 432-scenario missing-completely-at-random factorial, fsQCA calibrated with substantive 2/3/4 anchors recovered the programmed gate in 0.749 of replications and falsely detected the pair in 0.015 of non-gate replications. Raw-score NCA yielded rates of 0.673 and 0.087, respectively. NCA applied to 2/3/4 fuzzy memberships was conservative (0.049 recovery), whereas percentile-calibrated fsQCA was less specific (0.519 recovery; 0.162 false detection). In matched scenarios, increasing sample size improved semantic-anchor fsQCA recovery but reduced raw NCA recovery as observations filled the ceiling region. These results show that calibration anchors and NCA scope definitions are part of the estimand. Applied studies should preregister external anchors, report scope and algorithmic non-results, and test recovery against a plausible counterfactual data-generating process. The benchmark evaluates analytical workflows; it does not estimate how Vietnamese residents assess public services.

Research Square
Thuongmai University (VN), Trường Đại học Kinh tế và Quản trị kinh doanh (VN)
Qualitative Comparative Analysis Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon