ShapeBench: A new approach to benchmarking local 3D shape descriptors

The ShapeBench evaluation methodology is proposed as an extension to the popular Area Under Precision-Recall Curve (PRC/AUC) for measuring the matching performance of local 3D shape descriptors. It is observed that the PRC inadequately accounts for other similar surfaces in the same or different objects when determining whether a candidate match is a true positive. The novel Descriptor Distance Index (DDI) metric is introduced to address this limitation. In contrast to previous evaluation methodologies, which identify entire objects in a given scene, the DDI metric measures descriptor performance by analysing point-to-point distances. The ShapeBench methodology is also more scalable than previous approaches, by using procedural generation. The benchmark is used to evaluate both old and new descriptors. The results produced by the implementation of the benchmark are fully replicable, and are made publicly available. • The ShapeBench benchmarking strategy for testing local 3D shape descriptors. • The Descriptor Distance Index metric for contextualising descriptor distances. • A replicable implementation of the proposed benchmark. • An evaluation of the Spin Image, USC, RoPS, SHOT, RICI, and QUICCI descriptors.