Edge deployment of YOLOv12 and YOLOv26 instance segmentation for real-time steel surface defect inspection

Abstract Real-time automated steel surface inspection on the production line is challenging. Defect masks must be produced at the camera’s acquisition rate and within the power envelope of an embedded GPU. This paper evaluates two contrasting YOLO architectures, the attention-centric YOLOv12 and the convolution-centric YOLOv26, for instance segmentation of steel surface defects on three platforms: an RTX 5090 workstation, a Jetson AGX Orin, and a Jetson Orin Nano. Twenty-four models were trained in a $$2{\times }2$$ 2 × 2 ablation over weight initialization (scratch, pretrained) and class sampling (standard, balanced) on a real industrial dataset of 3,097 images and 13,318 instances across 8 classes (11-fold imbalance). The models were then evaluated under PyTorch and under TensorRT in 16-bit floating-point (FP16) and 8-bit integer (INT8) precision. Accuracy is reported on a leak-free test subset after a split-integrity audit. The best configuration, YOLOv26 pretrained with FP16, achieves 0.43 mean average precision (mAP50-95) at large scale. All YOLOv26 pretrained models surpass the 30 frames-per-second (FPS) real-time budget on both Jetson devices under FP16: 89.9 FPS at nano on the Orin Nano and 146.7 FPS on the AGX Orin. YOLOv12 also deploys under TensorRT FP16 on all three platforms and, trained from scratch, matches YOLOv26 pretrained accuracy at the medium scale. The training-time choices (architecture, initialization, and sampling) are statistically indistinguishable; therefore, deployment is the deciding factor. FP16 preserves accuracy within 0.02 mAP50-95. However, INT8 segmentation engines fail to build on both Jetson devices under TensorRT 10.3 and their accuracy is unreliable on the workstation.