MDC-Net: multimodal defect captioning network for surface steel defects

Abstract During steel production, ensuring the integrity of the product’s surface is crucial for maintaining a competitive edge. Detecting surface flaws is a key component in preserving high standards of production quality, affecting both the end product’s reliability and the efficiency of the manufacturing process. Traditional methods for identifying these defects have relied heavily on manual inspection or basic computer vision techniques, both of which present a significant challenge in terms of accuracy and the health and safety of the inspectors. This paper explores an innovative approach by integrating a sequence generation model incorporating the transformer architecture, enhancing the detection of defects in the production of quality hot-rolled steel sheets. This method, which solves object detection through the lens of sequence generation, offers a more intricate analysis of images, enabling a comprehensive examination of surface anomalies along with detailed annotations regarding their nature and precise location. The approach not only proposes to increase the accuracy of defect detection but also highlights its adaptability for broader industrial uses.