Principal Curvatures as Pose-Invariant Features of Depth Maps for RGB-D Object Recognition

Computer vision tasks, such as object recognition, using deep learning find their place in a variety of contexts including agriculture. Regarding data, the coupling of RGB and depth modalities has already proven to be beneficial for object recognition over the use of RGB-only images. However, the lack of neural network architectures and large-size datasets dedicated to the depth modality forces us to use backbones pre-trained on RGB data using large datasets such as ImageNet. While works proposed by Eitel et al. and Aakerberg et al. rely on colorizing the depth values to match an RGB format, they do not take full advantage of the geometric properties carried by the depth modality. We demonstrated principal curvatures when used to color-encode the depth values retain more information related to the object's shape. The proposition was evaluated on the Washington RGB-D dataset and gave mitigated results mainly explained by a high confusion between similarly shaped objects, which represent an important fraction of the dataset. With the introduction of superclasses based on the geometric shape of objects (sphere, cylinder, cube,... ) our model performed higher than the previous work, e.g. 3.1% precision increase for the sphere superclass. While presenting some limitations, this work opens the path for further developments.