A convolutional neural network-based joint detection and localization spatiotemporal scheme for process control through speckle pattern imaging

• An AI-based control scheme for process monitoring using speckle images. • A novel approach for joint detection and localization of shifts in the speckle patterns. • A CNN architecture tailored for speckle pattern monitoring. • A CNN training methodology adapted to speckle images. • Extensive simulation studies with comparisons to existing non-AI methods. This paper presents a novel convolutional neural network (CNN)-based control chart for detecting localized and subtle speckle shifts (displacements) in the images. To enhance detection accuracy, especially in detecting localized and simultaneous shifts, we design a custom CNN architecture and utilize a training methodology. Moreover, instead of inputting the entire images, we divide them into equal-sized grids and process them as two-channel inputs: one containing the shifted grids and the other capturing the difference between shifted and reference grids. A maximum probability-based control scheme is developed to concurrently detect shifted images and localize shifted regions within an image. To the best of our knowledge, this is also the first spatiotemporal scheme that simultaneously performs detection and localization, while conducting image-based process control. We perform simulation studies under two main scenarios: (i) spatial domain analysis and (ii) frequency domain analysis. The control chart’s performance is evaluated across various grid sizes, shift magnitudes, and shifted areas. Additionally, we demonstrate our scheme’s capability through several real-time monitoring examples. Comparative analysis with the existing methods showed that our CNN-based control chart significantly outperformed the existing approach, particularly in detecting small global shifts, localized shifts, and simultaneous shifts.