Event-based vision in laser welding: An approach for process monitoring

Event cameras differ from conventional frame cameras: Rather than capturing images at a fixed rate, they asynchronously measure pixelwise brightness changes and stream an event sequence that includes time, location, and sign of the brightness changes. Event cameras provide attractive properties such as high temporal resolution and latency (both in the order of ms), very high dynamic range (120 dB), and low power consumptions because no routine operation needs to be recorded. Hence, event cameras have been successfully used, for instance, for object tracking, pose estimation, or image deblurring. Common in-process sensors for process monitoring in laser welding are photodiodes or high-speed frame cameras. Due to the described advantages of event cameras compared to these sensors, this paper for the first time investigates the potential of event cameras in laser welding. We present different promising ways to process the unconventional asynchronous output of event cameras for process monitoring, including time-series representations, as well as image representations. Our laser welding experiments show clear differences for different production scenarios in those event representations. Additionally, we propose a machine learning pipeline for the automatic detection of production errors with event camera.