Is facial beauty in the eyes? A multi-method approach to interpreting facial beauty prediction in machine learning models

Despite advances in facial beauty prediction, how specific facial regions contribute to perceptions of attractiveness remains largely unexplored, highlighting a critical interpretability gap in this domain. This study addresses the interpretability gap in facial beauty prediction (FBP) models by introducing a novel framework that combines global and local interpretability methods. We introduce Region Attribution, a technique that aggregates XRAI (eXplanation with Ranked Area Integrals) saliency maps across predefined facial regions to quantify their relative importance in individual predictions. Two global approaches complement this local interpretability: permutation feature importance, which systematically explores individual facial regions across the dataset to measure performance degradation, and individual feature prediction, where separate CNN models are trained on isolated facial regions to assess their independent predictive power. Using the SCUT-FBP5500 and MEBeauty datasets, we train convolutional neural networks on both full faces and individual facial features. While our findings reveal slight variations in feature rankings across the three methods, they consistently identify the eyes and nose regions as crucial determinants in facial beauty prediction. Thus, this study demonstrates the value of a multi-method approach in understanding the complex interplay of facial features in beauty prediction machine learning models.