An Importance Sampling Monte Carlo Framework for Fan Vote Reconstruction and Voting Rule Evaluation: A Case Study of Dancing with the Stars

In television dance competitions such as Dancing with the Stars (DWTS), fair and transparent outcome determination is crucial for addressing public concerns regarding contest fairness. However, existing studies face challenges due to the opacity of fan voting and the lack of a systematic evaluation of rule changes, making fairness disputes difficult to resolve. To address these challenges, this study develops a data-driven counterfactual analysis framework to quantitatively assess the impact of different voting rules on competition outcomes. We develop a constraint-based Monte Carlo framework to characterize feasible fan-vote distributions consistent with historical elimination results, together with an IS-MC variant for controlled reconstruction comparisons. A paired semi-synthetic validation comprising 1145 trials across five fan-vote-generating mechanisms, with the generated vote shares concealed during reconstruction, showed that IS-MC achieved the lowest mean MSE (0.01915) and MAE (0.08980), together with the highest mean Spearman correlation (0.382) and empirical 95% interval coverage (0.879) among the compared methods under a common sampling budget. Its lower ESS-to-valid-sample ratio nevertheless suggests a trade-off between reconstruction quality and importance-weight stability. Within the modeled counterfactual comparisons, the percentage-based rule was more sensitive to variation in fan-vote proportions, and contestants whose reconstructed fan support was high relative to their judge-score shares tended to receive more favorable modeled rankings under that rule. Under the stylized bottom-two judge-selection scenario, the modeled outcomes differed when judges retained the contestant with the higher judge score; this is a conditional sensitivity finding rather than evidence about actual judges’ decisions. Additionally, a six-feature GBDT proxy was evaluated using chronologically held-out percentage-rule observations, and SHAP analysis on the predicted-probability scale was used to characterize feature associations with modeled counterfactual reversals. We construct an end-to-end system encompassing data preprocessing, model inversion, uncertainty quantification, semi-synthetic validation, and diagnostic visualization, providing a descriptive basis for comparing DWTS voting rules. The proposed methodological framework can also be extended to other competitive scenarios relying on hybrid decision-making.