BabyFM: towards accurate 3D baby facial models using spectral decomposition and asymmetry swapping

Abstract

In this paper, we present the first publicly available 3D statistical facial shape model of babies, the Baby Face Model (BabyFM). Constructing a model of the facial geometry of babies entails specific challenges, such as occlusions, extreme and uncontrollable expressions, and data shortage. We address these challenges by proposing (1) a non-template dependent method that jointly estimates a 3D facial baby-specific template and the point-to-point correspondences; (2) a novel method to establish correspondences based on the spectral decomposition of the Laplace Beltrami Operator, which provides a more robust theoretical foundation than state-of-the-art methods; and (3) an asymmetry-swapping strategy to alleviate the shortage of large scale datasets by decoupling the identity-related and the asymmetry-related shape deformation fields. The latter leads to a data augmentation technique that we integrate within the Gaussian Process Morphable Model framework, providing a simple way of combining synthetic or sample covariance functions. We exhaustively evaluate each stage of our method and demonstrate that (1) when aiming at the 3D facial geometry of a baby, a specific model of babies is needed, since the pre-built publicly available models constructed with adults or older children are not able to accurately represent the facial shape of babies; (2) our spectral approach improves correspondences accuracy with respect to state-of-the-art-methods; and (3) the proposed data augmentation technique enhances the robustness of the BabyFM.

This work is partly supported by MICIU/AEI/10.13039/501100011033/ under the project grant PID2020-114083GB-I00 and PRE2021-097544 scholarship, the ICREA Academia programme and the NIH Eunice Kennedy Shriver National Institute of Child Health & Human Development grant R42 HD08171203.