Dual-Vision Robot Safety Early Warning System

Abstract This paper introduces a dual-vision robot safety early warning system leveraging 3D vision to safeguard against unauthorized personnel approaching operational robots. Utilizing two Time-of-Flight (TOF) cameras, the system semantically segments robots from their surroundings and evaluates nearby personnel's status. It generates control commands to maintain a safe distance, crucial for industrial environments with human-robot collaboration. The system addresses challenges posed by the unordered nature of 3D point clouds by employing a re-projection-based segmentation algorithm. It re-projects point clouds onto a 2D plane, annotates the robot, trains a semantic segmentation network, and projects results back into the point cloud. However, this method may be limited by the 2D image angle, and segmentation accuracy may be affected for complex scenes or occlusions. Future work will explore the integration of multi-view 2D images or the use of advanced 3D segmentation techniques to address these limitations. For efficient image segmentation, the system uses ResNet18, a lightweight network suitable for speed-critical tasks, with pre-trained weights to boost training performance. Although ResNet18 improves segmentation speed, the entire system still relies on high-performance computer hardware for model training and real-time processing. To further reduce computing resource consumption, the use of lightweight models or model pruning technology will be considered in future work. The system's reliability is validated through experiments simulating random human approaches to the robot's safety zone, demonstrating its effectiveness in real-time safety monitoring. This research contributes an efficient method for ensuring robot safety in industrial settings, enhancing human-robot interaction safety.