Unlocking 3D baby face photogrammetry: Multi-view BabyMorph reconstruction from uncalibrated photographs

• First multi-view infant 3D face reconstruction algorithm from triplets images (frontal, left and right) • Transformer-based multi-view 2D-3D reconstruction network • Geometric deep learning autoencoder replaces PCA-based models for richer, nonlinear facial shape representation • Cost-effective and simple approach, suitable for limited-resource settings Craniofacial anomalies are important diagnostic markers in early life. Recent studies emphasize the value of 3D imaging for extracting robust facial features that are potential indicators of disease. However, widespread availability of 3D scanning devices in hospitals remains a challenge and this type of technology may not be available in limited-resource settings. For this reason, we present a new approach to generate precise baby 3D face reconstructions from multiple uncalibrated 2D photographs acquired with a smartphone camera. The novel multi-view transformer network presented takes as input three uncalibrated photographs of the baby in frontal, left, and right pose. It then maps these images to a previously learned latent space that captures the baby’s 3D facial morphology, using a 2D vision transformer encoder. Subsequently, the estimated 3D geometry is recovered by decoding the latent vector using a 3D graph convolutional network decoder. Our network demonstrates a normalized mean error of 3.29% and a root mean square error of 2.62 mm between reconstructed and true 3D faces in the baby test dataset. These outcomes are comparable with those reported by both single-view and multi-view 2D-3D reconstruction state-of-the-art errors in adult models. To conclude, the presented Multi-view BabyMorph generates highly accurate 3D baby facial reconstructions from uncalibrated photographs, simplifying the process of 3D photogrammetry. This innovation expands access to advanced baby diagnosis tools, particularly in resource-limited settings.