Uncertainty driven pose estimation - for rigid objects in CNN-based pipelines

Nowadays, image-based object recognition and pose estimation are highly active research areas due to their importance in robotic perception and interaction. While modern CNN-based pose estimators achieve great results, they lack transparency regarding the trustworthiness and precision of individual estimates. This lack of certainty inhibits further processing of the results and deters deployment in production environments due to reliability concerns. As an answer, this thesis proposes a fusion-based approach in which, due to a novel output architecture, the CNN self-estimates the amount of information obtained, resulting in individual 6D uncertainty estimates per 6D pose estimate. Specifically, the CNN predicts the observed object points pixel-wise, along with the precision in the image plane of those predictions. All such gathered perspective information is then fused (without linearization) into a single, globally valid 13 × 13-sized information matrix, which is then regressed to yield the six-dimensional result. This separation allows the CNN to operate solely in image space, whereas the conversion from 2D image space to 6D pose is solved analytically. Additionally, the intermediate result of the globally valid information matrix facilitates the fusion with auxiliary information, such as depth, stereo, and prior knowledge, with ease, as it is simply a 13 × 13 matrix addition. With this approach, the pose is regressed from a fusion of all available data, unlike the more ad hoc approach of combining estimates in postprocessing. Also, the CNN call is wholly unaffected by the addition of these supplemental data. An extensive evaluation of the proposed architecture on multiple benchmark datasets showcases meaningful uncertainty estimates while maintaining competitive pose performance. Also, it shows that adding auxiliary information can significantly improve pose performance, but always relative to the amount of new information gained while maintaining the quality of the estimated uncertainty.