Fig 1.
Example of a dataset of 1000 samples from Gaussian distribution with normal boundary arbitrarily established.
Anomaly detection methods identify outliers as anomalies and calculate their distance from the center of the distribution.
Fig 2.
In order to develop an automated facial scoring system several steps were required: Image preprocessing, face normalization, color transformation, heat map calculation, morphological erosion and anomaly scoring.
Fig 3.
The objective of the proposed method is to obtain a machine score that aligns closely with a human rating by utilizing a difference map generated by comparison of original image with its normalized counterpart.
Fig 4.
Applying different preprocessing steps to generate consistent images with the StyleGAN2 pretrained model.
This includes adjusting the background, scale, and rotation/flip orientation.
Fig 5.
The proposed normalization algorithm searches for the latent vector most closely matching the input face.
The latent vector is then frozen and the StyleGAN2 model is optimized to reconstruct more facial details without anomalies. G refers to the StyleGAN2 generator network.
Fig 6.
Facial transformation by applying the face inversion and model adaptation algorithms sequentially.
The face in green border is the one that is taken as the best normalized version of xorg showing distinct features of the face without showing the anomaly. Having more adaptation iterations will reverse the normalization.
Fig 7.
Transforming the facial image from RGB to YCbCr.
Note that most of the variation is in the Y component. Cb and Cr exhibit negligible variation.
Fig 8.
Comparison of various heatmaps generated using LPIPS, SSIM, and the proposed PSE method.
The PSE approach demonstrates a more localized anomaly signal.
Table 1.
Pearson correlation coefficient between the human and machine scores of the 61 cleft faces under analysis, utilizing different combinations of heatmap generation and image processing approaches.
The best combination is highlighted in bold below.
Table 2.
Pearson correlation coefficient between the human and machine scores of the 125 StyleGAN2-generated and real cleft faces under analysis, utilizing different combinations of heatmap generation and image processing approaches.
The best combinations is highlighted in bold numbers.
Fig 9.
The Pearson correlation between human and machine scores of the 125 faces under study using different algorithmic approaches.
The optimal heatmap and imaging processing combination for each of the 3 approaches is depicted here (as reflected in bold in Table 1) Note tighter alignment of human/machine scoring with the PSE method.
Table 3.
Average evaluation time per sample in seconds for each combination of operations under analysis, when xorg and xnorm are provided.
Another 135 seconds have to be added for each normalization step with adaptation to obtain xnorm, while 123 seconds have to be added when using the standard StyleGAN2 normalization procedure.
Fig 10.
If we keep adapting the StyleGAN2 generator for more than 50 iterations, the model starts reconstructing the abnormal details of the original face.
Here the cleft anomaly gradually appears in the generated face.