Robot-relay: building-wide, calibration-less visual servoing with learned sensor handover networks

Abstract This paper proposes a novel system to conduct visual servoing of a mobile robot using multiple uncalibrated, wall-mounted cameras. Specifically, we utilise a constellation of such sensors to cover a wide area by allowing robot control to be passed between cameras in regions where their fields of view overlap. This method, in conjunction with the fact that all computing is also executed offboard, allows for simpler and cheaper robots to be deployed in controlled and finite spaces. Our method simplifies the natural installation cycle of a newly deployed camera network, eliminating the need for explicit camera positioning or orientation, both globally (relative to a building plan) and locally (among viewpoints). Our system memorises pixel-wise topological connections between viewpoints by leveraging natural human exploration of the environment. We detect graph edges through simultaneous detections of the same person across different cameras, allowing us to automatically construct an evolving graph that represents overlapping fields of view within the camera network. In combination with a hybrid-A*-based planner, our approach allows efficient planning and control of robots across a wide area by traversing cameras between areas of overlap. We validate our approach through autonomous traversals in a productive office environment, using a network of six cameras, and compare our performance against both human teleoperation and a traditional Simultaneous Localisation and Mapping (SLAM) approach.