Relative pose estimation from panoramic images using a hybrid neural network architecture

Camera-based relative pose estimation (RPE) localizes a mobile robot given a view at the current position and an image at a reference location. Matching the landmarks between views is critical to localization quality. Common challenges are appearance changes, for example due to differing illumination. Indirect RPE methods extract high-level features that provide invariance against appearance changes but neglect the remaining image data. This can lead to poor pose estimates in scenes with little detail. Direct RPE methods mitigate this issue by operating on the pixel level with only moderate preprocessing, but invariances have to be achieved by different means. We propose to attain illumination invariance for the direct RPE algorithm MinWarping by integrating it with a convolutional neural network for image preprocessing, creating a hybrid architecture. We optimize network parameters using a metric on RPE quality, backpropagating through MinWarping and the network. We focus on planar movement, panoramic images, and indoor scenes with varying illumination conditions; a novel dataset for this setup is recorded and used for analysis. Our method compares favourably against the previous best preprocessing method for MinWarping, edge filtering, and against a modern deep-learning-based indirect RPE pipeline. Analysis of the trained hybrid architecture indicates that neglecting landmarks in a direct RPE framework can improve estimation quality in scenes with occlusion and few details.