Deployment Validity of a StitchingNet-Lineage Sewing Defect Classifier: A Cross-Domain, Cold-Start, and Edge-Deployment Evaluation
High accuracy on a held-out test split from the training distribution is a necessary but insufficient condition for factory-floor reliability, since production lines encounter fabric-thread, lighting, and hardware conditions that a single-domain benchmark does not exercise. This study evaluates that gap for a MobileNetV1-based sewing-stitch defect classifier built on a published stitching-defect dataset and pipeline that reported an in-distribution macro-F1 of 0.99. The model was retrained and evaluated along five deployment-relevant axes: leave-one-domain-out (LODO) generalization, cold-start recovery, robustness to eight input perturbations, calibration reliability, and edge-device feasibility after TensorFlow Lite quantization. The retrained model reproduces the original in-distribution result closely (accuracy 0.9804, macro-F1 0.9805). Under a zero-effort LODO protocol, performance falls to a mean accuracy of 35.4% (macro-F1 32.9%) across eleven domains and three seeds, and a repeated-measures analysis of variance confirms that the spread is driven by domain rather than seed. Matching the LODO fine-tuning budget to the in-distribution schedule for four domains does not close this gap, indicating a genuine domain-shift limitation rather than under-training. A cold-start budget of fifty labeled images per class, on a query set held fixed across budgets, recovers most of the lost performance (macro-F1 91.2%). The model is well calibrated in-distribution (expected calibration error 0.0055) but degrades sharply under LODO shift (0.30 to 0.82 across four domains), failing confidently rather than only inaccurately. Quantization reduces model size by 91.3% at a cost of 1.17 accuracy points. For this lineage and dataset, an in-distribution F1 of 0.99 does not imply deployment readiness.
Authors
- Hamdan S. Bintang (ORCID: https://orcid.org/0009-0002-5976-2213)
- Fajar Pitarsi Dharma (ORCID: https://orcid.org/0000-0003-3473-3432)
Institutions
- Akademi Komunitas Industri Tekstil dan Produk Tekstil Surakarta (ID)
Publication Details
- Journal
- Journal of Computers Mechanical and Management
- Published
- 2026-10-31
- DOI
- https://doi.org/10.57159/jcmm.5.5.261421
- Primary Topic
- Industrial Vision Systems and Defect Detection
- Type
- article
- Field-Weighted Citation Impact
- 0.00