Differences in eye movements for face recognition between Canadian and Chinese participants are not modulated by social orientation

Francis Gingras; Amanda Estéphan; Daniel Fiset; He Lingnan; Roberto Caldara; Caroline Blais

doi:10.1371/journal.pone.0295256

Abstract

Face recognition strategies do not generalize across individuals. Many studies have reported robust cultural differences between West Europeans/North Americans and East Asians in eye movement strategies during face recognition. The social orientation hypothesis posits that individualistic vs. collectivistic (IND/COL) value systems, respectively defining West European/North American and East Asian societies, would be at the root of many cultural differences in visual perception. Whether social orientation is also responsible for such cultural contrast in face recognition remains to be clarified. To this aim, we conducted two experiments with West European/North American and Chinese observers. In Experiment 1, we probed the existence of a link between IND/COL social values and eye movements during face recognition, by using an IND/COL priming paradigm. In Experiment 2, we dissected the latter relationship in greater depth, by using two IND/COL questionnaires, including subdimensions to those concepts. In both studies, cultural differences in fixation patterns were revealed between West European/North American and East Asian observers. Priming IND/COL values did not modulate eye movement visual sampling strategies, and only specific subdimensions of the IND/COL questionnaires were associated with distinct eye-movement patterns. Altogether, we show that the typical contrast between IND/COL cannot fully account for cultural differences in eye movement strategies for face recognition. Cultural differences in eye movements for faces might originate from mechanisms distinct from social orientation.

Citation: Gingras F, Estéphan A, Fiset D, Lingnan H, Caldara R, Blais C (2023) Differences in eye movements for face recognition between Canadian and Chinese participants are not modulated by social orientation. PLoS ONE 18(12): e0295256. https://doi.org/10.1371/journal.pone.0295256

Editor: Antoine Coutrot, CNRS: Centre National de la Recherche Scientifique, FRANCE

Received: April 24, 2023; Accepted: November 18, 2023; Published: December 14, 2023

Copyright: © 2023 Gingras et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Data is available at the following OSF location: https://osf.io/b5tdy/

Funding: This work was supported by grants from the Natural Sciences and Engineering Research Council of Canada (NSERC; https://www.nserc-crsng.gc.ca/) to C.B. (RGPIN-2019-06201) and D.F. (RGPIN-2012-402513), and by a Canada Research Chair in Cognitive and Social Vision (# 950-232282) held by C.B. (https://www.chairs-chaires.gc.ca/chairholders-titulaires/profile-eng.aspx?profileId=4770#:∼:text=Caroline%20Blais%2C%20Canada%20Research%20Chair,aims%20to%20increase%20this%20understanding.) The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Perception is shaped by experience and adaptation to the environment [1]. Our visual and social surroundings are molded by distinct cultural norms and values. Culture can be viewed as a vehicle of tradition of thought through which interpretation of the world is formed. During the last two decades, growing evidence arising from different fields has shown that interiorization of cultural norms plays a role in shaping perception [2, 3]. These findings have demonstrated that mechanisms of perception are not universal, as they are intrinsically linked to cultural backgrounds and social structures specific to geographical locations. This constructivist way of envisioning perception has given rise to many cross-cultural studies in this field, spanning from optical illusions [4] and color perception [5] to facial expression recognition [6]. Culture might determine the way we structure ourselves, how we interact with the world and how we perceive the environment.

It has been demonstrated that face recognition strategies do not generalize across individuals [7–10] and are reliable over time [11]. Notably, studies have revealed robust differences between individuals from East Asian and West European/North American countries ranging from eye movement strategies and information sampling resolution (i.e. spatial frequencies) during face recognition to emotion perception [3, 6, 12]. Specifically, for face recognition, evidence suggests East Asians adopt a more global attentional strategy involving more central fixations [13]. This has been related to peripheral processing of facial features [14, 15] and lower spatial frequency use in East Asian populations [10, 16, 17]. On the other hand, West European and North American individuals adopt a more local attentional strategy, involving fixations towards facial features and higher spatial frequency sampling. These results are important, as face-to-face contacts play a central role in social interactions. Nevertheless, exactly what cultural factors are responsible for such perceptual differences remains to be understood.

The social orientation hypothesis posits individualistic vs. collectivistic (IND/COL) value systems [18–21] as a major factor of explanation for many cultural differences in visual perception [2, 22] (however, see [23, 24]). This framework also includes the concept of independent and interdependent self-construals, as studies have shown that independent self-construal correlates with individualism and interdependent self-construal with collectivism [25]. This theory suggests that collectivistic values, more prevalent in East-Asians, would be linked to a holistic cognitive style, while individualistic values, more common in North Americans/West Europeans, would be linked to an analytic cognitive style. In terms of visual perception, this would translate into broader and narrower visual attention, respectively. This view is supported by evidence of differences between East Asians and West Europeans/North Americans along the dimensions of Individualism/Collectivism and Independence/Interdependence [19, 25], that are associated with differences in visual attention patterns [22].

For instance, when asked to judge the orientation of a line placed inside an independently rotating frame, Chinese participants are more influenced by the orientation of the frame than American participants [26, 27]. Later studies highlighted that Japanese participants were also more influenced by a changing background when asked to memorize foreground objects in a scene [28, 29]. The latter differences in scene memorization between Japanese and American participants were further supported by divergent eye movement patterns. In that study, Chinese participants gazed at the background for a longer period and started directing fixations to foreground objects later than American participants [30–33].

However, there have also been studies showing evidence that the social orientation hypothesis may not be what drives the observed differences between groups. Knox & Wolohan [23] have compared Chinese individuals raised in England to both English and Chinese participants, and have shown that while they are closer to British participant in their values, their eye movement patterns are more similar to those of the Chinese group. Another study by Kardan et al. [24] comparing collectivist Mayan to US participants has shown that participants from the US fixate more on the background information during videos compared to Mayan children, which is not what would be expected with the social orientation hypothesis. While the authors do advise caution in applying the social orientation theory to their findings (it is unclear whether the individualism-collectivism continuum applies to Mayan culture), it still shows that not all of the influences attributed to culture are as clear-cut as they were expected to be.

Some studies have sought to more directly link social orientation to visual processing. Namely, a causal link between social orientation and perception was demonstrated using independent and interdependent priming [34]. The study results showed that “independence primed” participants were quicker to identify the small letters compared to the large letters in Navon figures, and the opposite was true for “interdependent primed” participants. Another study by Chua et al. [35] as shown that independent self-construal predicts worse performance in holistic perceptual learning. These studies suggest that social orientation plays a critical role in shaping cultural perceptual differences between West European/North American and East Asian observers, whereby collectivism/interdependence is associated with global attention and individualism/independence is associated with local attention.

Despite the aforementioned evidence, whether social orientation can also explain the cultural differences in face recognition remains unclear. A recent study using social self-construal priming on a Chinese population found that some instances of face processing, such as holistic processing and race categorization, were indeed affected by social self-construal priming, but that face recognition was not [36]. However, this study only included a sample of Chinese participants, limiting the generalizability of its findings and their genuine attribution to cross-cultural causes. Another study, with a sample of British-born Chinese participants, has highlighted a possible link between individual social values and facial recognition strategies. In their research paper, Kelly et al. [37] report a non-significant but considerable relationship between individualism/ collectivism and eye movement patterns during face recognition. Individualistic individuals were more likely to exhibit a fixation pattern typically associated with West Europeans and North Americans, whereas collectivistic individuals were more likely to exhibit a fixation pattern typically associated with East Asians. However, the sample size was quite small in those groups, likely not enough for sufficient statistical power, and only one questionnaire was used to measure individualism/collectivism.

To clarify these important issues, we investigated whether social orientation could account for cultural differences in eye movements during face recognition. We conducted two experiments to test if both primed and self-reported social orientation modulate eye movements during an Old/New face recognition task [13, 38]. To be consistent with previous social orientation studies, we have chosen designs that are similar to previous studies, both in the methods and questionnaires used. In Experiment 1, we probed the existence of a link between individualism/collectivism (IND/COL) and eye movements during face recognition by implementing an IND/COL priming paradigm. We used a within-subjects design as they have been shown to have better statistical power in priming paradigms [39]. This method is similar to the one used in Liu et al. [36]. However, unlike the latter study, we included both West European and Chinese participants.

In Experiment 2, we investigated the relationship between social orientation and facial recognition in greater depth by using two IND/COL questionnaires and including subdimensions to those concepts. We used the validated Auckland Individualism and Collectivism Scale (AICS), developed by Shulruf et al. [40], as well as a validated shortened version of the Horizontal and Vertical Individualism and Collectivism Scale (HVICS), originally developed by Triandis & Gelfand [41]. Both questionnaires were validated in the target populations [42, 43] This method is akin to the protocol used in Kelly et al. [38]. Once again, we included participants from both cultures (North America and China).

Culture is a broad term that refers to a wide variety of concepts (ex. social norm, upbringing, belief systems, geographic location, etc.). Previous research has shown that country borders represent a relatively good proxy of cultural variations [44, 45]. In line with this idea, in the present article we define groups of individuals coming from East-Asian and Western countries as “culturally different”, in the sense that they have grown up in environments with different sets of values, social norms, and so on.

Experiment 1

General information

West European and North American participants had little to no experience with East Asian cultures; Chinese participants had little to no experience with West European and North American cultures. To assess participants’ experience with other cultures, we asked them to indicate if they had ever lived in a country other than their birth country. All participants were aged between eighteen and thirty-five years old and had normal or corrected-to-normal vision. Participants were recruited between 2015 and 2018. Task instructions, priming texts, and questionnaires were given to participants in their native language. Experimental protocols used in this study were approved by the Institutional Review Boards of University of Fribourg (Switzerland), Université du Québec en Outaouais (Canada), and Sun Yat-Sen University (China). Participants gave their written, informed consent to participate in the study and signed the consent form prior to the experiment. The entire study was conducted in conformity with the Code of Ethics of the World Medical Association (Declaration of Helsinki). Participant data was tied to an anonymous ID after data collection. Data analyses were conducted without any information, other than this ID, that could be linked to a participant’s identity.

Hypothesis

In line with Liu et al. [36], we hypothesized that group differences based on social orientation should mirror the differences between East Asians and West Europeans typically observed in face recognition strategies. Observers primed with a collective self-construal should exhibit more central fixations (i.e., nose area) whereas observers primed with an independent self-construal should exhibit more featural fixations (i.e., eyes and mouth areas).

Participants

Fifteen Swiss individuals and twenty-one Chinese individuals took part in the experiment. Swiss participants were tested in Fribourg (Switzerland) and were born and lived their entire life in Switzerland. Chinese participants were tested in Guangzhou (China) and were born and lived their entire life in China. Our sample size was selected according to a G*Power assessment for a Repeated measures ANOVA with a within-between interaction, given a medium effect size (f = .25), a minimal statistical power of 80%, α = .05, nb groups = 2 and nb measurements = 2. Other parameters were kept at the default values. A minimum sample size of 34 participants was recommended from these settings. This power assessment was made for the cultural group x priming condition mixed ANOVA and the effect size was based on previous studies that found large Cohen’s d effect sizes [13, 46]. Here, we used a smaller effect size than what was found in those studies with respect to cultural differences in eye movements during face recognition.

Material and stimuli

Images were 382×390 pixels in size and displayed with a gray screen-wide background on an 800×600 pixel Dell P11302100 (refresh rate of 170 Hz). Participants were seated at a viewing distance of 70 cm from the computer screen, stimuli subtending 15 degrees of visual angle horizontally.

The face dataset used in this study is the one used in Blais et al. [13] There were a total of 280 identities with an equal number of males and females. Each identity featured three facial expressions (happy, disgusted and neutral). The decision to include different facial expressions replicates Blais et al. [13] and limits the use of “template-matching” to identify faces. Participants were explicitly asked to focus on memorizing the identity and then recognize the face regardless of the displayed facial expression. White European face images were drawn from the KDEF database [47], and East Asian face images were drawn from the AFID database [48]. Faces from both databases faced forward. Accidental local features such as brown spots, rashes and facial hair were removed using the Photoshop software. Faces had no distinctive features such as glasses, jewelry, scarf, etc. All faces were aligned as well as possible, using as parameter the least-square measure, on the positions of eyes, nose and mouth–by means of translation, rotation, and scaling. Face images were all gray scale and normalized in terms of luminance. In order to maintain some degree of ecological validity, faces were otherwise presented with their normal hair and facial contour.

Eye-tracking apparatus

Eye movements were recorded at a sampling rate of 1000 Hz with the SR Research Desktop-Mount EyeLink 2K eye-tracker (average gaze position error of .25 degrees; spatial resolution of .01 degrees); only the dominant eye was tracked. In both experiments, participants’ dominant eye was determined using a variation of the Miles test [49], similar to the “hole in the hand test [50]”. To ensure that viewing distance was constant throughout the entire experiment and that eyes were accurately tracked, participants were asked to position their head on a chin and forehead rest, facing the screen at the appropriate viewing distance.

A nine-point manual calibration of ocular fixations was performed before each task block (see protocol section for details regarding “blocks”). Calibrations were then validated with the EyeLink software and repeated when necessary until the optimal calibration criterion was reached. At the beginning of each trial, participants were instructed to fixate a dot at the center of the screen to perform a drift correction. The experiment was implemented in MATLAB R2012b, using the Psychophysics [51–53] and EyeLink [54] toolboxes (see http://psychtoolbox.org).

Protocol

An Old/New face recognition task along with an “Individualist (I) / Collectivist (We)” pronoun circling priming task [55] was used for this experiment. Overall, the experimental procedure went as follows.

Participants had to complete four “learning / recognition” blocks. Before each block, participants had to read one of four priming texts that were either individualist-oriented (written with first person singular pronouns, e.g. “I”) or collectivist-oriented (written with first person plural pronouns, e.g. “We”). Ultimately, all participants read four texts, two individualist-oriented (once before learning White European faces; once before learning Chinese faces) and two collectivist-oriented (once before learning White European faces; once before learning Chinese faces), and thus all were subjected to both priming conditions (i.e., “I” or “We”-oriented texts). Half of the participants were primed with the two “I” texts first, the other half with the two “We” texts first. The order of face ethnicities was counterbalanced across participants. Participants were asked to carefully read each priming text. As a means of priming, they were instructed to circle all pronouns included in the text. To make sure participants paid close attention to the textual content, they were informed that they had to answer questions about the story once they finished.

During learning periods, participants were instructed to memorize a series of fourteen different faces, presented to them one at a time for five seconds. During recognition periods, participants were presented with a second series of faces, half of which were previously memorized during the corresponding learning period, and were instructed to use the keyboard to indicate whether each face was one they had seen before (old; ‘A’ button on keyboard) or not (new; ‘L’ button on keyboard). Once again, faces were displayed one by one, but this time appeared on screen until a response. Faces appeared in the corners of the screen, so that first fixations were voluntarily aimed at a specific point in the face, instead of being centered on the initial fixation point. See Fig 1 for an illustration of the Old/New experimental design.

Download:

Fig 1. Old/New experimental design.

Old/New face recognition experimental design and examples of stimuli: a) learning period (sequence of 14 faces per block; presentation time of 5 seconds per face); b) recognition period (sequence of 28 faces ‐‐ 14 new ‐‐ per block; presentation until response).

https://doi.org/10.1371/journal.pone.0295256.g001

Analyses

Fixation maps were produced for each participant and each experimental condition (i.e., priming situation). Fixation maps were computed by using fixation durations along image-centered coordinates. Fixation durations across individual maps were transformed into z-scores using the map fixation duration mean and standard deviation. For both experiments 1 and 2, we chose to analyze all eye movements produced during learning (5 seconds) and recognition (until response), again following the procedure of Blais et al. [13].

Learning and recognition periods were always analyzed separately. Eye movements were first analyzed using the iMap4 Linear Mixed Model [56] (LMM). To obtain the LMM parameters, iMap4 performs a pixel-by-pixel comparison of the fixation maps across participants and conditions. Then, a bootstrap procedure is applied to correct for multiple comparisons (i.e., across the pixels of the fixation maps). A fixed effect mixed ANOVA with “culture” as between factor and “priming” as within factor was performed on the resulting maps (alpha = .05; bootstraps = 1000). “Face ethnicity” was not considered since no effect of this variable was found in the original Blais et al. [13] study, as well as many subsequent replication studies [10, 14, 16, 38] (however, see [57, 58]).

As a second step, we investigated the effect of culture and priming on specific face regions where significant differences were previously found between West Europeans and East Asians. To this effect, we conducted a Region Of Interest (ROI) analysis on the corresponding eyes, nose, and mouth regions of the fixation maps; a mean of the face stimuli used for the experiment was taken as a template to delimit the ROIs (see Fig 2). These regions were selected as they are important in distinguishing respectively West European fixation patterns (feature-oriented) and East Asian fixation patterns (center-oriented). We also included in our analysis a ROI corresponding to the rest of the face, around the eyes, nose, mouth, as was done in Blais et al. [13]. Percentage of fixations allocated to the eyes, nose, mouth, and rest of the face respectively were compared between both cultural groups (Swiss and Chinese), and between both priming conditions (“I” and “We” personal pronoun priming). To examine group differences on fixation percentages for each ROI, we performed “Culture” x “Priming” ANOVAs on each ROI separately. The bilateral significance threshold (p < .025) was corrected for multiple comparisons using a Bonferroni correction, where differences on each ROI are tested at p = .025/4 = .006.

Download:

Fig 2. Face ROI template.

Note. Average face stimulus with ROI corresponding to eyes (red), nose (blue), mouth (yellow), and rest of face (green).

https://doi.org/10.1371/journal.pone.0295256.g002

Results

Culture, social orientation priming, and eye movements

Fixation maps (see Fig 3) show that there is a trend toward the typically observed cultural differences in fixation patterns (more fixations on eyes for West Europeans; more fixations on center for East Asians), especially for learning periods. As for “priming conditions”, on the other hand, no trend is observed (see Fig 4).

Download:

Fig 3. Fixation maps–cultural differences.

Note. First column illustrates Swiss participants’ average fixation maps for learning and recognition periods respectively; second column illustrates Chinese participants’ average fixation maps for learning and recognition periods respectively; third column illustrates Swiss (yellow) and Chinese (blue) fixation biases for learning and recognition periods respectively (no significant cultural differences were revealed by iMap4). The term “fixation bias” refers to fixation locations that are associated more with one group compared to the other.

https://doi.org/10.1371/journal.pone.0295256.g003

Download:

Fig 4. Fixation maps–priming differences.

First column illustrates the “I” personal pronoun priming group’s average fixation maps for learning and recognition periods respectively; second column illustrates the “We” personal pronoun priming group’s average fixation maps for learning and recognition periods respectively; third column illustrates the “I” priming group (yellow) and “We” priming group (blue) fixation biases for learning and recognition periods respectively (no significant priming group differences revealed by iMap4).

https://doi.org/10.1371/journal.pone.0295256.g004

The mixed ANOVA (culture x priming) analysis with iMap4 yielded no significant results. This means that pixel-by-pixel differences in fixation maps for the main effect of culture, the main effect of priming and the culture x priming interaction were not revealed at least 95% of the time. In addition, mixed ANOVAs yielded no main effect of priming on any ROI, and no significant interactions between culture and priming. See Table 1 for detailed ROI-related statistics.

Download:

Table 1. Mixed ANOVAs with culture x priming on ROI (fixation percentages).

https://doi.org/10.1371/journal.pone.0295256.t001

Interestingly, there was only an effect of culture on fixations over the rest of the face which was marginal for learning [F(1,34) = 7.590; p = .009; η2p = .182] and significant for recognition [F(1,34) = 9.230; p = .005; η2p = .213] periods. Independent Samples t-tests confirmed that Chinese participants distributed their fixations in the “rest of face” ROI more than Swiss participants during learning [t(34) = -2.750; p = .009; Cohen’s d = -.931] and recognition [t(34) = -3.040; p = .005; Cohen’s d = -1.030] periods. Average fixation percentage differences between cultures and between priming conditions are reported in Table 1 for each ROI.

Discussion

In experiment 1, we revealed significant cultural differences that were atypical compared to previous results, and priming did not seem to have any effect on participant. It is unclear whether this is because there is no effect of priming or because the priming procedure simply failed. Our data show a qualitative cultural contrast, with more fixations towards the eyes for West Europeans and central fixations for East Asians. Analyses with iMap4 yielded no significant results. However, subsequent ROI analyses did reveal greater fixation distribution around the face for Chinese participants compared to Swiss participants. This supports the idea that Chinese observers spend more time than Swiss observers fixating areas peripheral to the main features. However, it differs from previous studies in that Chinese and Swiss participants did not differ in the time spent fixating the main facial features nor the center of the face. This difference may be due to the fact that our Chinese sample was recruited from a different region than the one from Blais et al.[13]. This is still consistent with the idea proposed by Caldara et al. [14], Miellet et al. [15], Tardif et al. [10] and Estéphan et al. [16] that East-Asians process facial features in peripheral vision. Further investigation would be necessary to examine how samples collected from different regions within the same country may show different patterns of visual strategies.

Since iMap4 is especially conservative compared to the versions used in previous studies, the General Linear Model might have downplayed the effect of culture considering the presence of other variables in the model with the present data. Moreover, the absence of robust significant differences between our cultural groups (with iMap4) might be related to individual differences in our sample, which could have been amplified by the “priming conditions” causing partial, but non-significant, interference. This observation emphasizes the importance of further investigating how individual social values could be associated with facial recognition strategies, which will be central to Experiment 2.