Auditing VirusTotal consensus thresholds in malware benchmarks: A reproducible security-measurement study on EMBER2024

VirusTotal (VT) consensus is widely used to construct malware benchmarks, yet the consensus threshold is usually fixed without sensitivity analysis. We treat this threshold as a benchmark-design variable, not a new detector. On EMBER2024, released binary labels remain fixed; raising the threshold T only removes low-consensus malware from the training pool. A fixed-label audit sweeps T from the released default 0.065 to 0.50 across four classifier families, an 11-point LightGBM grid, calibration, fixed-form weighting, file-type analyses, and three control arms, totaling 600 model fits over ten seeds. LightGBM identifies an empirical positive effect region T ∈ [0.08, 0.18], peaking at 𝑇 = 0 . 1 2 with a 4.73 percentage point gain in challenge TPR@1%FPR (Cohen’s 𝑑 = 2 . 0 3 , Holm-adjusted 𝑝 = . 0 2 0 ) and 5–8 × lower seed dispersion; XGBoost shows the same coarse-grid pattern. A matched-size control finds no discrimination gain under random removal; random removal accounts for about one fifth of the ECE reduction. ECE decreases through this region but continues improving after discrimination degrades, so calibration alone does not select an operating point. All eight tested fixed-form weighting schemes reduce low-consensus-malware detection. The numerical range is specific to EMBER2024, a nominal denominator of approximately 77 engines, and gradient-boosted classifiers. The reusable contribution is the audit procedure and panel-relative reporting rule 𝑘 = ⌈ 𝑇 ⁢ 𝑁 ⌉ ; each new corpus requires re-audit against labels fixed independently of the candidate training thresholds.