Spherical Harmonics for Robust Next-Best-View Estimation

As 3D depth cameras become more affordable, the additional depth information proves useful in industrial setups. However, with a multiview camera configuration, the complete coverage of a 3D model can only be accurately captured from several views. For this task, the next-best-view (NBV) estimation is essential to minimize the number of captions required for a complete reconstruction. Traditional NBV methods for 3D data are often affected by sensitivity to noise and are not rotation invariant. Learning-based methods can make estimations based on current camera information and significantly improve a task’s accuracy. These methods are generally limited to one step, as multi-step planning requires estimating sensor observations. A method that can predict and simulate multiple steps can significantly improve performance where one step is insufficient.We show that a convex hull of the object can be used to predict n-steps of NBV in a coverage reconstruction setting. The convex hull is used as a proxy for the real geometry of the object in the selection of the views and requires much fewer points sampled from the ground-truth geometry. We propose a volumetric information gain (VIG) score that measures the similarity between the convex hulls obtained from the partial point cloud from selected views and the ground truth point cloud densely sampled from the geometry.We retrain an NBV network to predict the views that lead to a point cloud with the highest VIG score and show that spherical harmonic preprocessing can make the network faster. Once the views for the convex hull have been selected, we show that spherical reconstruction of the partial point clouds can further compress the relevant geometric information into a much smaller set of coefficients. This method is robust to changes in view space, as shown in the tests applied on Shapenet models along with more complex synthetic shapes from HomebrewDB [1], Stanford 3D Scanning Repository [2], and Linemod [3].