The role of uncertain perspective information in recovering 3D symmetrical shapes

It is known that 3D shape reconstruction from a single 2D perspective image of a symmetrical object is unique, but 3D reconstruction from an orthographic image yields a one parameter family of possible 3D shapes. Last year, we showed that subject’s perceived 3D shape from a perspective image tended to be closer to veridical when compared to reconstruction from an orthographic image. In our new experiment, we tested how the human visual system incorporates perspective information. On each trial in the experiment, a perspective image is shown on the left side of a monitor, and on the right side of the screen a subject-adjustable 3D shape is shown rotating. The subject adjusted the aspect ratio of the mirror-symmetrical rotating 3D shape on the right until the 3D reconstruction was as close as possible to their 3D percept produced by the static 2D image on the left. From trial to trial, object shown and simulated distance were adjusted. As the distance of an object becomes greater and the retinal image becomes smaller, perspective information becomes less reliable, and the subject’s percept becomes less veridical. Within this framework, we tested two classes of objects: (i) synthetic symmetrical polyhedral objects and (ii) symmetrical or approximately symmetrical, real-world objects selected from the ModelNet-40 dataset. The computational model uses regularization to perform 3D shape reconstruction. The cost function has terms that penalize departure from 3D compactness of the 3D shape’s convex hull. In addition, the cost function incorporates a measure of the reliability of perspective information. The cost function biases the 3D reconstruction towards veridicality when perspective information is reliable. With subjects, reconstruction of natural objects is more accurate than reconstruction of synthetic polyhedral objects. This result suggests the presence of additional shape constraints, such as a second mirror symmetry.