A deep learning-based multi-modal approach to robust place recognition in challenging orchard environments

Place recognition addresses the problem of identifying the current location on revisit using the acquired knowledge of all the previously explored locations. The abstract representations of the locations can be generated as they are being explored using processed single or multi-sensory inputs. Later, the current abstraction of the environment can be compared with previously collected information to detect revisits. The loop closure component of Simultaneous Localization And Mapping (SLAM) uses these techniques to identify revisits, which allow the SLAM system to mitigate the associated long-term drift. RGB images, LiDAR point clouds, and LiDAR intensities are the three most popular sensory inputs used in place recognition literature. However, they each have certain disadvantages when used as a single-modal input. Additionally, in recent years, Deep Neural Network (DNN) based methods have emerged increasingly in the literature that addresses the place recognition problem. Therefore, In this work, we introduce a novel multi-modal method that takes advantage of the rich complementary information provided by the above three modalities along with a DNN for place recognition. This work was evaluated on multiple publicly available datasets as well as on a highly repetitive orchard dataset collected by our team. The results demonstrate the ability of this method to be used even in highly challenging environments such as orchards.