Human-like face pareidolia emerges in deep neural networks optimized for face and object recognition
Fig 2
Experimental methods and analyses.
A We compared the neural MEG responses with representations in CNNs using the 96 images from Wardle et al. [12]. The figure displays iconic placeholders representing the three different stimulus categories (i.e., faces, pareidolia, and matched objects), along with an example image used in the experiments. The icons used in this figure have been obtained from The Noun Project (https://thenounproject.com/) under the royalty-free license. Note that we do not have the right to display the human face images used in the experiments. The face image example shown in this figure is a similar photograph taken of one of the authors, who has granted permission for the publication of his identifiable image. The pareidolia and object images were sourced from Wardle et al. [12] (https://www.nature.com/articles/s41467-020-18325-8) and are used under a Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/). For legal compliance, parts of the images containing logos or brands have been covered with a white box. All stimuli used in the experiments are available at the Open Science Framework (https://osf.io/9g4rz). B We used five task-optimized CNNs based on the VGG16 architecture, each trained on different combinations of face and object tasks: 1. Dual-task CNN: trained on face identification and object categorization; 2. Face-identification CNN: trained solely on face identification; 3. Face-identification-and-Object-detection CNN: trained on face identification and object detection; 4. Object-categorization-and-Face-detection CNN: trained on object categorization and face detection; and 5. Object-categorization CNN: trained solely on object categorization. C To generate a representational dissimilarity matrix (RDM), we initially passed the stimulus set through each CNN and extracted activations from specific layers of the network to obtain feature vectors. We then computed pairwise correlations between all feature vectors and subtracted them from 1, resulting in layer-wise RDMs. D We used multidimensional scaling (MDS) to visualize the obtained layer-wise RDMs. E By using representational similarity analysis (RSA), we measured the similarity (i.e., Spearman correlation between the upper triangles of the RDMs) between the layer-wise RDMs and the RDMs derived from different time steps of neural MEG data and idealized model RDMs.