Automatic exposure strategy network for robust visual odometry in environments with high dynamic range

Abstract The success of visual-based localization requires the image features to be sufficiently captured in well-exposed images and reliably tracked among image sequences. In this paper, a learning-based exposure strategy algorithm is developed to improve the performance of visual odometry or simultaneous localization and mapping in scenes with sudden illumination changes and high dynamic range (HDR). The exposure value is predicted by optimizing a novel image information metric, which is calculated based on a multi-scale image histogram. Experimental observations show that the proposed metric is highly related to the localization performance, and achieves better localization accuracy when compared to other state-of-the-art metrics. Using this novel metric, the exposure control network is designed to predict the exposure value for the next image given a sequence of recently captured images as inputs. For different application scenarios, alternative network frameworks are proposed with the options to include an optimization module providing improved exposure values for feedback learning, but with additional computation complexity. The approach is light-weight for real-time video applications and the images captured using the predicted exposure values are well-exposed with better detected and matched features in a variety of challenging HDR scenes. Code will be available in https://github.com/leejieyu13/HDR-AE .