Deployment Validity of a StitchingNet-Lineage Sewing Defect Classifier: A Cross-Domain, Cold-Start, and Edge-Deployment Evaluation

High accuracy on a held-out test split from the training distribution is a necessary but insufficient condition for factory-floor reliability, since production lines encounter fabric-thread, lighting, and hardware conditions that a single-domain benchmark does not exercise. This study evaluates that gap for a MobileNetV1-based sewing-stitch defect classifier built on a published stitching-defect dataset and pipeline that reported an in-distribution macro-F1 of 0.99. The model was retrained and evaluated along five deployment-relevant axes: leave-one-domain-out (LODO) generalization, cold-start recovery, robustness to eight input perturbations, calibration reliability, and edge-device feasibility after TensorFlow Lite quantization. The retrained model reproduces the original in-distribution result closely (accuracy 0.9804, macro-F1 0.9805). Under a zero-effort LODO protocol, performance falls to a mean accuracy of 35.4% (macro-F1 32.9%) across eleven domains and three seeds, and a repeated-measures analysis of variance confirms that the spread is driven by domain rather than seed. Matching the LODO fine-tuning budget to the in-distribution schedule for four domains does not close this gap, indicating a genuine domain-shift limitation rather than under-training. A cold-start budget of fifty labeled images per class, on a query set held fixed across budgets, recovers most of the lost performance (macro-F1 91.2%). The model is well calibrated in-distribution (expected calibration error 0.0055) but degrades sharply under LODO shift (0.30 to 0.82 across four domains), failing confidently rather than only inaccurately. Quantization reduces model size by 91.3% at a cost of 1.17 accuracy points. For this lineage and dataset, an in-distribution F1 of 0.99 does not imply deployment readiness.

Authors

Institutions

Publication Details

Journal
Journal of Computers Mechanical and Management
Published
2026-10-31
DOI
https://doi.org/10.57159/jcmm.5.5.261421
Primary Topic
Industrial Vision Systems and Defect Detection
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Deployment Validity of a StitchingNet-Lineage Sewing Defect Classifier: A Cross-Domain, Cold-Start, and Edge-Deployment Evaluation

Hamdan S. Bintang, Fajar Pitarsi Dharma
Journal of Computers Mechanical and Management
Industrial Vision Systems and Defect Detection
article

Deployment Validity of a StitchingNet-Lineage Sewing Defect Classifier: A Cross-Domain, Cold-Start, and Edge-Deployment Evaluation

Hamdan S. Bintang, Fajar Pitarsi Dharma
article en

Abstract

High accuracy on a held-out test split from the training distribution is a necessary but insufficient condition for factory-floor reliability, since production lines encounter fabric-thread, lighting, and hardware conditions that a single-domain benchmark does not exercise. This study evaluates that gap for a MobileNetV1-based sewing-stitch defect classifier built on a published stitching-defect dataset and pipeline that reported an in-distribution macro-F1 of 0.99. The model was retrained and evaluated along five deployment-relevant axes: leave-one-domain-out (LODO) generalization, cold-start recovery, robustness to eight input perturbations, calibration reliability, and edge-device feasibility after TensorFlow Lite quantization. The retrained model reproduces the original in-distribution result closely (accuracy 0.9804, macro-F1 0.9805). Under a zero-effort LODO protocol, performance falls to a mean accuracy of 35.4% (macro-F1 32.9%) across eleven domains and three seeds, and a repeated-measures analysis of variance confirms that the spread is driven by domain rather than seed. Matching the LODO fine-tuning budget to the in-distribution schedule for four domains does not close this gap, indicating a genuine domain-shift limitation rather than under-training. A cold-start budget of fifty labeled images per class, on a query set held fixed across budgets, recovers most of the lost performance (macro-F1 91.2%). The model is well calibrated in-distribution (expected calibration error 0.0055) but degrades sharply under LODO shift (0.30 to 0.82 across four domains), failing confidently rather than only inaccurately. Quantization reduces model size by 91.3% at a cost of 1.17 accuracy points. For this lineage and dataset, an in-distribution F1 of 0.99 does not imply deployment readiness.

Journal of Computers Mechanical and ManagementVol. 5(5)
Akademi Komunitas Industri Tekstil dan Produk Tekstil Surakarta (ID)
Openalex Percentile: Top 13%
Industrial Vision Systems and Defect Detection
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon