Learning of Multimodal Point Descriptors in Radar and LIDAR Point Clouds

Registration of point clouds is a fundamental task in robotic SLAM pipelines. Typically this task is performed only on point clouds of the same sensor or at least the same sensing modality. However, robots designed for challenging environments are often equipped with redundant sensors for the same task where some sensors are more accurate and others are more robust against disturbing environmental conditions. Being able to register the data across the modalities is an important step to more fault-tolerant localization and mapping. We therefore propose a learning framework, which describes the points in the point cloud invariant of their modality. This description is then used in a transformer-like model to find point matches for the registration process. We demonstrate our results using a scanning lidar and radar sensor on our own and publicly available datasets.