A Vision-Based Software Safety Monitoring Tool for Operators in RoboDK Robotic Cells: A Simulation-Based Proof-of-Concept Study Using Workspace Masks and Image Processing

This article presents the development and proof-of-concept testing of a vision-based safety monitoring tool for operators in simulated robotic cells in RoboDK. The proposed method uses a virtual camera placed above the cell and image processing techniques to analyze the relationship between the operator and the workspace swept by the robot. In an initial stage, the robot movement is recorded as a mask of the swept space, and areas irrelevant to the process can be excluded by user-defined polygons. In the monitoring stage, the operator is identified in the video stream by HSV segmentation, after which an adjustable clearance zone is generated around the detected contour. Based on the intersections between the operator, clearance, swept space mask and the mask of the current robot movement, the application provides four discrete states: SAFE, WARNING, DANGER and COLLISION. For the experimental validation in the virtual environment, the virtual contact moment is estimated separately, while the COLLISION state is defined as the intersection between the inflated operator contour and the current robot motion mask. Therefore, in this study, COLLISION does not indicate measured physical contact, but an image-based imminent-collision condition used for early warning. The test scenario was carried out in a virtual palletizing cell, which includes an articulated arm robot, conveyors, manipulated objects and a mobile dummy acting as an operator. The obtained results support the use of the method as an applicative simulation solution for the evaluation of the early detection of risk situations. The study is limited to the virtual environment and represents a basis for future research on the development of visual monitoring systems to increase safety in collaborative and industrial robotic cells.