Online Vision-Based 3D Reconstruction for Shared Human-Robot Workspaces

In applications where robot arms collaborate with humans, the corresponding shared workspaces are very dynamic and not as predictable as purely robotic ones. Unknown obstacles, such as arbitrary boxes or tools placed by humans, might appear at any time and require the robot to react accordingly. Often, expensive Lidar- or depth-sensors are applied in these scenarios. Instead, we propose an approach to check for obstacles and prevent potential collisions in the corresponding workspace that is entirely based on less expensive RGB cameras. The workspace is monitored with an extrinsically calibrated multi-view camera setup. Our proposed hybrid approach combines learning-based feature matching and traditional image processing methods to achieve online 3D reconstruction. Even less textured objects and surfaces can be perceived and modelled as simplified collision objects. We present several experiments and show the effectiveness of learning-based feature matching and the advantages of combining them with accurate computer vision methods. Further, we show that our approach runs in near real-time on our demonstrator using a multi-view setup with three cameras.