Reconstructing Detailed 3D Car Geometries from a Single 2D Image by Repurposing PIFu

User-generated content is an integral part of the Internet. With VR and AR technologies becoming more accessible and affordable, there will be an increase in demand to create 3D content independently. Through AI, the process of creating 3D content for the average user can be made easier. In this study, we will focus on PIFu, one of the most prominent 2D to 3D reconstruction methods. PIFu is an end-to-end deep learning reconstruction method that uses an implicit representation that locally aligns the pixels of the 2D image with the global context of its corresponding 3D object. Using the same method, it can also infer the texture colours of the object. PIFu was originally designed for reconstructing 3D geometry of humans wearing clothes along with their intricate 3D textures, such as folds and creases. In this study, we tested the capability of PIFu in reconstructing non-human geometries. We trained this model on the car object categories to reconstruct detailed 3D car geometries. We evaluated the test results for twenty-four geometries, resulting in <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{0.046} \pm \mathbf{0.019}\ \mathbf{m},\ \mathbf{0.343}\pm \mathbf{0.118}\ \mathbf{m}$</tex>, and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{0.677}\pm \mathbf{0.124}$</tex> for average symmetric surface distance, Hausdorff distance, and intersection over union, respectively. Visually, the geometry also has pronounced details, most notably the clear separation between the wheels and the body.