Design and evaluation of deep learning-based arbitrary multiple camera simultaneous localization and mapping system

Abstract Visual simultaneous localization and mapping (SLAM) systems are extensively explored due to their low cost. To tackle more challenging environments, multi-camera SLAM systems with a wider field of view (FoV) have been proposed. However, most of the existing multi-camera SLAM systems still rely on hand-crafted features. Given the remarkable advancements of deep learning techniques in the field of feature extraction, the integration of convolutional neural networks into arbitrary multi-camera SLAM systems represents a research topic worthy of in-depth exploration. In this paper, a Deep learning-based Arbitrary Multi-Camera SLAM system (DAMC-SLAM) is proposed. Specifically, this system leverages convolutional neural networks (CNNs) to perform online extraction of local features and global descriptors, while generating corresponding masks to filter out dynamic objects in the scene. Furthermore, DAMC-SLAM is designed with strong adaptability, as it can be flexibly compatible with an arbitrary number of cameras, regardless of whether there exist overlapping FoVs between the deployed cameras. Six self-collected sequences under various environmental conditions, together with the EuRoC MAV benchmark, were used to evaluate the performance of DAMC-SLAM. Experimental results demonstrate that our system possesses remarkable robustness and superior performance. Compared with other methods, it achieves consistent and successful tracking performance across all self-collected sequences, while concurrently yielding the minimum absolute trajectory error and scale error over these sequences. It also demonstrates competitive performance on the EuRoC MAV benchmark, achieving the lowest average ATE among the evaluated methods.