Evaluating Plug-and-Play Modifications in Lightweight Industrial Defect Detection: An Empirical Anatomy and Statistical Bounds on NEU-DET

Lightweight object detectors frequently integrate plug-and-play modules (e.g., attention mechanisms, modified bounding box losses) into industrial defect detection pipelines. Evaluations of such variants, however, often overlook seed-to-seed variance and split-induced volatility. We benchmark YOLOv8n and YOLO11n on the corrected NEU-DET dataset across repeated random seeds and a 5-fold cross-validation protocol, with the primary evaluation endpoint fixed before folds were executed. Neither ECA channel attention nor Wise-IoU loss yields a detectable gain: three-seed validation point estimates are negative (-0.0169 and -0.0069 [email protected]), and the 5-fold paired difference for Wise-IoU is -0.0030 ± 0.0132 (p = 0.638), inside the measured resolution floor of ±0.0165. Both signs replicate on the frozen held-out test split. Error anatomy on 3,316 ground-truth boxes indicates that failure modes are dominated by a 13.7-percentage-point generalization gap in recall (0.922 on seen vs. 0.785 on held-out images) rather than by classification-head confusion (0.1% of audited boxes). Test-time weighted box fusion provides an empirical variance-reduction upper bound (+0.0385), yet distilling it back into single models through consensus pseudo-labeling returns a null effect (+0.0018 ± 0.0071, p = 0.596). These findings delineate the measurement resolution of lightweight industrial defect detection.