Clean Accuracy Versus Robustness: Rank Shifts in CNN and Vision Transformer Document Classifiers

Purpose – Archival systems select document classifiers from clean-accuracy tables; however, repositories receive scans that are blurred, noisy, compressed, skewed, and unevenly lit. Whether document-specific pre-training buys robustness as well as clean accuracy is untested: no study has compared convolutional networks, generic vision transformers, and document-specific transformers under one graded corruption protocol on all 16 RVL-CDIP classes. Design/methods/approach – ResNet-50, EfficientNet-B0, ViT-Base, and DiT-Base were fine-tuned on a balanced RVL-CDIP subset under an identical protocol, with only the learning rate tuned per architecture family, and then evaluated across nine archival corruptions at five severities. Retention is reported beside absolute drop, a 0.75 clean-accuracy gate confined the comparison to competent systems, and McNemar’s test supplied significance. Findings: All four cleared the gate (clean accuracy 0.7956–0.8706), yet the rankings diverged. DiT-Base was most accurate on clean input, while ViT-Base was most robust, retaining 0.9478 against 0.8673 and shedding 0.0430 accuracy points against 0.1155. Both ends of the reversal were significant (p = 5.30 × 10−8 clean; p = 4.99 × 10−108 under severe blur). Illumination and blur drove the gap; a supporting SHA-256 Merkle layer flagged every simulated tamper. Research implications/limitations – The findings rest on a single seed, a subsampled training set, and simulated rather than genuinely scanned degradation; therefore, the ranking describes the configuration evaluated here rather than the architecture families in general. Originality/value – The study answers an architecture-level question that earlier work could not reach: clean accuracy is an unsafe sole criterion for selecting classifiers in archival systems.