Facial masks affect emotion recognition in the general population and individuals with autistic traits

Facial expressions, and the ability to recognize these expressions, have evolved in humans to communicate information to one another. Face masks are equipment used in healthcare by health professionals to prevent the transmission of airborne infections. As part of the social distancing efforts related to COVID-19, wearing facial masks has been practiced globally. Such practice might influence affective information communication among humans. Previous research suggests that masks disrupt expression recognition of some emotions (e.g., fear, sadness or neutrality) and lower the confidence in their identification. To extend the previous research, in the current study we tested a larger and more diverse sample of individuals and also investigated the effect of masks on perceived intensity of expressions. Moreover, for the first time in the literature we examined these questions using individuals with autistic traits. Specifically, across three experiments using different populations (college students and general population), and the 10-item Autism Spectrum Quotient (AQ-10; lower and higher scorers), we tested the effect of facial masks on facial emotion recognition of anger, disgust, fear, happiness, sadness, and neutrality. Results showed that the ability to identify all facial expressions decreased when faces were masked, a finding observed across all three studies, contradicting previous research on fear, sad, and neutral expressions. Participants were also less confident in their judgements for all emotions, supporting previous research; and participants perceived emotions as less expressive in the mask condition compared to the unmasked condition, a finding novel to the literature. An additional novel finding was that participants with higher scores on the AQ-10 were less accurate and less confident overall in facial expression recognition, as well as perceiving expressions as less intense. Our findings reveal that wearing face masks decreases facial expression recognition, confidence in expression identification, as well as the perception of intensity for all expressions, affecting high-scoring AQ-10 individuals more than low-scoring individuals.

Introduction in high autistic trait individuals. Therefore, in the current study we investigate the effect of facial masks on FER across three experiments using different populations (university students and the general population), and individual difference measures (low and high scorers on an autism quotient scale). Specifically, in Experiment 1 and using a large sample of university students, we test the effect of facial masks (or simply, masks) on FER, as well as the confidence in the recognition of the expressions and the perceived intensity level of those expressions. While recent studies have examined the effect of masks on FER [16,[18][19][20][21][22], only one [22] investigated participants' confidence in their FER abilities with masked faces, and none of the studies asked participants to rate the intensity of emotions in masked conditions. In Experiments 2, we replicate the first experiment using a broader sample from the general population by recruiting participants from Mechanical Turk workers. Finally, in Experiment 3 we investigate the effects of facial masks as a function of high and low scores on the 10-item Autism Spectrum Quotient (AQ-10 [44]).

Experiment 1
In the first experiment, we recruited university students to test the effect of masks and the sex of the stimuli on FER. We predict that facial masks will lead to a general decrease in FER across all emotions.  13.87, p < .001); and better in recognition of masked male emotions than female masked emotions (z = 7.35, p < .001; see S1 Table in S1 File for the frequency and percentage of responses for each facial expression). No difference was observed for the correct identification between unmasked male and unmasked female facial expressions (z = 2.39, p = .101).
Confidence in facial expression recognition. A series of generalized linear mixed models were conducted to investigate the effect of facial mask on the confidence in FER. Each of the expressions were added as a fixed factor and participants as a random factor. Results showed significant main effects of Mask for confidence in all expressions (Anger: β = 0.12, SE = 0.01, χ 2 (1) = 3463. 22

Discussion
Using a sample of university students, the effect of facial masks on FER was investigated and the results showed that participants were better in FER of unmasked faces than masked faces. Furthermore, participants were better in FER in male masked faces than in female masked faces. Participants were less confident in their recognition of all investigated expressions behind the masks than facial expressions without masks. The overall confidence in FER was also sensitive to the sex of the faces, as participants were more confident in recognition of anger, disgust, fear, and neutrality in male faces, but more confident in recognition of happiness and sadness in female faces. When faces were masked, participants were more confident in their identification of anger and fear in male faces than in female faces, while they were more confident in identifying sadness in masked female faces than in masked male faces. Yet, all of the emotions were rated as less expressive when under a mask. While male faces were rated as more expressive in anger, disgust, fear, and happiness, and female faces were more expressive in sadness, participants rated fear and happiness in masked male faces as more expressive than in masked female faces, and rated sadness in masked female faces as more expressive than in masked male faces.
Using a database similar to the one used in the current study, Carbon [22] did not find a significant difference between masked and unmasked fear and neutral expressions. One potential explanation for this difference between our study and Carbon's concerns the number of stimuli used and participants tested. While in the Carbon study 41 individuals rated the emotions of 12 identities (a combination of young, middle-aged, and older faces), we recruited 420 participants to identify the emotions of 16 young identities. Our larger population, as well as our larger set of stimuli within a coherent age group, may have yielded a more robust effect, revealing that recognition of all emotions, including fear and neutral faces, are affected by wearing a mask. Moreover, by using a larger sample size, our analyses had the power to test for the interactions between mask and stimulus sex on FER. However, there are still limitations to our study, for example, using a sample of only university students. Therefore, in the next study we aimed to test the effect of facial masks on FER within a more general population.

Experiment 2
Experiment 2 tested if the results of Experiment 1 generalise to a broader population. We recruited participants from Amazon Mechanical Turk workers.

Method
Participants. A total of 199 individuals (130 men and 69 women), aged between 18 and 73 years (M = 34.58, SD = 10.18) were recruited through Amazon Mechanical Turk (MTurk) from the United States to complete an online survey. Participants provided online consent to participate. MTurk's robustness is extensively researched and results support it as a valid means of collecting online data for the behavioural sciences, sometimes even superior to inperson collection [46]. MTurk subjects are often more representative of the U.S. population than in-person convenience samples, but they might be less representative than subjects in Internet panels or national probability sampling [46][47][48]. A total of 141 participants (70.9%) reported being married, and 3.5% as being divorced, widowed, or separated; an additional 19.6% reported being single, and 6.0% in a relationship. In terms of their highest academic degree, 10.1% had a high school diploma, 6.0% had a post-secondary diploma, 47.7% of the participants had an undergraduate degree, and 35.7% had a post-graduate degree (MA or PhD).
Stimuli and procedure. Twelve young faces, six male and six female, with six emotional expressions (anger, disgust, fear, happiness, sadness, and neutral) were obtained from the FACES database [45], resulting in 72 stimuli. Another set of 72 stimuli were created by superimposing a facial mask on the original photos. The faces used in this experiment were a subset of faces used in Experiment 1. The reduction of stimuli (a total of 192 trials in Experiment 1 to 144 in Experiment 2) was to reduce the length of the study and the risk of participants dropping out [49,50]. A within-subjects design was used, and participants randomly observed either the block with facial masks first or the block without the masks first. The rest of the procedure, including the questions asked, was as in Experiment 1.

Facial expression recognition.
A generalized linear mixed model was conducted to investigate the effect of facial masks, and stimulus sex on the percentage of correct facial expression recognitions, with participant as a random factor. Once again, due to the imbalance of males in our sample, we did not consider participant gender as an effect in our analysis.    Fig 5). No significant difference was observed for intensity in anger and happiness between unmasked male and female faces (Anger: z = -0.76; Happiness: z = 1.54; ps > .747), or between unmasked and masked sad female faces (z = -1.75; p = .479).

Discussion
The second experiment aimed to test if the results of Experiment 1, which used a university student sample, replicate in the general population. Results from Mechanical Turk workers showed that similar to Experiment 1, participants' FER decreased significantly as a function of facial mask, and that participants were less confident in their recognition of the facial expressions behind the masks. This significant difference was observed for all expressions. Moreover, all facial expressions under the masks were rated as less expressive than those without masks, and male faces were considered more expressive in anger, disgust, and happiness compared to female faces.

Experiment 3
Our final study examined the effect of a facial mask on FER for high autistic trait individuals. Given that individuals with autistic traits place more importance than NTs do on the mouth to recognize facial expressions, we predict that a facial mask will make FER particularly difficult for people with autistic traits. Specifically, they will be less accurate than NTs at FER, and that masks will negatively impact their FER accuracy more than they do NTs'.

Method
Participants. One hundred and forty-two participants were recruited from the Human Subject Pool at the University of British Columbia according to the school's ethical guidelines, and received extra credit for agreeing to participate. Participants provided online consent to participate. All participants completed the 10-item Autism Spectrum Quotient (AQ-10; [44]) prior to the experiment, and they had no way of knowing that their AQ-10 score was relevant to the present study. The AQ-10 includes 10 items from the 50-item Autism Spectrum Quotient (AQ; [26]), and was designed to be a quick and effective screening method for ASD. The AQ-10 scores reliably in specificity (0.91), sensitivity (0.88), and has a positive predictive value of 0.85 [44]. An individual who scores 6 or higher on the AQ-10 is referred for a full diagnostic assessment. For individuals with ASD, the AQ-10 yields a mean score of 7.93, and for NTs a mean score of 2.77 [44]. Thus, higher scores on the AQ-10 indicate more autistic traits have been endorsed.
Participants were divided into two groups: "high scorers", comprised of participants who scored 6 or higher on the AQ-10, and "low scorers", which included participants who scored 5 or lower on the AQ-10. This division yielded 71 participants (50 women Stimuli and procedure. The same stimuli and procedures were used from Experiment 1.

Facial expression recognition.
A linear generalized mixed model was conducted to investigate the effects of AQ-10 (high vs. low score) and masks on the percentage of correct facial expression recognitions, with participants as a random factor. As in the previous two experiments, we did not consider participant gender as an effect in our analysis due to the imbalance of males in our sample. Results showed significant main effects for Mask and AQ-10, however their interaction was not significant The results of post hoc comparison for the effect of masks on the degree of drop in FER confidence showed a greater reduction in confidence for fear, neutral, and sad faces for high scorers compared to low scorers. The mean difference in confidence ratings between masked and unmasked sad faces was 12.71 for high scorers, but only 9.78 for low scorers. Similarly, results show that high scorers' mean difference between the two conditions for neutral faces was 10.83, but only 6.82 for low scorers. The reduction in confidence between unmasked and masked fearful faces for high scorers was 10.26, while only 8.48 for low scorers.
Intensity of expression. To investigate the effect of facial masks on the ratings of intensity of facial expressions, a series of generalized linear mixed models were conducted with each of the expressions as a fixed factor and participants as a random factor. Note that the question of    43, 77.29], z = 3.15, p = .010), respectively. Also, high scorers perceived male faces as more expressive in disgust than female faces (z = 11.91, p < .001; Fig 8). No difference was observed between low and high scorers for confidence in sadness for both masked (z = 2.46; p = .083) and unmasked face (z = 2.02; p = .258).
The results also returned significant three-way Mask × AQ-10 × Stimulus Sex interactions for fear and happiness

Discussion
Testing the effect of facial mask on FER in low and high AQ-10 scorers showed that high scorers (those who endorsed more autistic traits) had less accuracy in FER than low scorers (those who endorsed few autistic traits), which supports previous research (deficits in identifying anger, disgust, fear [30,51], sadness, surprise [30], and happiness [51]). Additionally, high scorers were less confident than low scorers in recognizing emotions labeled as anger, disgust, happiness, and neutral in unmasked faces, while they were less confident in recognizing masked faces labeled as disgust and neutral compared to low scorers. Furthermore, high scorers rated all of the emotions as less intense than did low scorers. They also rated male faces displaying disgust and fear, as well as masked female faces displaying happiness, as less intense than did low scorers. Similar to the overall results in the previous experiments, high scorers on the AQ-10 were better at FER in male faces than in female faces, and were more confident in the recognition of anger in male faces compared to anger in female faces. While both high and low scorers' confidence dropped significantly from unmasked to masked emotions of fear, neutral, and sad faces, the drop in confidence trended towards being larger for high scorers. Additionally, similar to low scorers, high scorers performed best when identifying unmasked faces labeled as happy. This supports previous reports that people with ASD are most accurate when identifying faces expressing happiness [30].
Recall that we hypothesised that high scorers on the AQ-10 would show deficits in FER when compared to low scorers. The results showed this to be the case. However, we also predicted that the introduction of masks would impede FER in high scorers more than they would in low scorers. This was not supported by the data. This lack of support is unexpected given that prior research has found that people use the mouth to identify happiness [52] and disgust [9], the eyes to discern fear and sadness [9], and that individuals with ASD look less at the eyes of fearful or neutral faces [53]. While masks caused a greater drop in confidence for high scorers than low scorers in expressions labeled as fear, neutral, and sad, there was no significant interaction between AQ-10 scores and masks for FER, despite high scorers rating both the intensity of emotions and their confidence in their FER ability lower than low scorers.

General discussion
Since the outbreak of COVID-19, health organizations have encouraged the general public to wear facial masks in order to curb its spread. While masks help combat against infection, they also hinder day-to-day social interactions, with previously unknown impact on individuals with autistic traits. As these individuals have been shown to have deficits in reading the emotions of others, the introduction of masks in daily social interactions was anticipated to have a greater impact on them than neurotypicals (NTs). Across three experiments we investigated the influence of masks on facial emotion recognition (FER). We tested the effect of masks on recognition of anger, disgust, fear, happiness, neutrality, and sadness in male and female faces using different samples (university students in Experiment 1, and the general population in Experiment 2), as well as examined the role of autistic traits (scores on the 10-item Autism Spectrum Quotient). We also assessed the confidence of participants in their judgements, and their perceived intensity of facial expressions. Overall our results showed that the introduction of masks disrupted participants' FER ability, their confidence in judging facial expressions, and their perception of emotion intensity.
Participants' accuracy in expression recognition decreased when faces were masked, a finding that was observed across all three studies, supporting previous research that reported similar findings [18,21,22]. Participants in the current study were also more accurate in FER when categorizing male versus female faces, although participants in Experiment 2 showed this pattern only in the masked condition.
Masks also affected confidence in FER. In all three experiments, participants were less confident in their judgements for all emotions in the mask condition compared to the unmasked condition, results that dovetail with the recent work by Carbon [22]. In the current study, participants were also less confident in identifying anger in female masked faces than male masked faces. For Experiments 1 and 2, participants were also less confident in identifying sadness in masked male faces than in masked female faces.
Because recent work has not addressed the effect that masks may have on the perception of emotional intensity, we asked participants to rate the level of expression (or intensity) in faces with and without masks. Across the three experiments, masks resulted in all facial expressions being perceived as less intense. These findings extend the research on the impact of wearing masks, beyond disruptions in FER accuracy and confidence. Sex differences in perception of expression intensity were also found: Experiments 1 and 2 returned higher ratings of sadness in masked female faces compared to masked male faces, and more intense perceptions of happiness for masked male faces than masked female faces. Additionally, participants in Experiment 2 perceived a higher intensity of anger in masked male faces than in masked female faces, while participants in Experiment 1 rated fear as more intense in masked male faces than in masked female faces.
Finally, Experiment 3 examined how traits of autism are related to perceptions of emotion expression. Recent research on COVID-19's impact on the autistic population has seen a focus on areas such as disruption of routines or services, how caregivers coped during lockdown, and mask tolerance training [54][55][56][57][58][59]. To extend the ASD and COVID-19-related research, we examined the effects of masks on possible social interactions for individuals with autistic traits in terms of FER, participants' confidence in their FER abilities, and their perception of facial emotion intensity.
Our results showed that compared to low scorers (participants who scored 5 or lower on the AQ-10), high scorers (those who scored 6 or higher) were significantly less accurate overall in FER, supporting findings from previous research [30]. However, we found no significant interaction between AQ-10 scores and masks. Given that masks cover the bottom half of the face, this result is counter to previous research that showed individuals with autistic traits preferentially look towards the mouth when viewing faces [35,36, but see 33], and that they rely more on the mouth region than NTs for categorizing emotions [38]. One possible explanation for our results may be rooted in Social Motivation Theory (SMT; [60]), which proposes that individuals with autistic traits can attend to salient social cues when prompted to do so, or if they believe doing so can help complete a task. The wearing of a mask, then, could serve as a visual prompt for autistic individuals to actively search for emotional clues elsewhere on the face.
To the best of our knowledge, this is the first study to examine the effect of facial masks on intensity ratings of expressions in autistic individuals' perception of emotion in the faces of others. However, our finding that high scorers rated the expressions in the stimuli as significantly less intense than low scorers does align with previous findings that autistic individuals have difficulty with rating emotions in unmasked faces [61]. Perceiving emotions as less intense, and therefore less informative, also aligns with our additional finding that high scorers reported being significantly less confident than low scorers in their ability to recognize anger, disgust, happiness, and neutral emotions in faces. Extending the research on autistic individuals' confidence in FER, we found their confidence trended lower for masked fear, neutral, and sad faces. We are unaware of previous studies examining autistic individuals' confidence in their FER abilities while viewing masked faces, although our results support Bekele et al. [62] who found autistic adolescents were less confident than controls (but similarly accurate) in FER in tasks involving unmasked faces. Note, however, that Sawyer et al. [63], found autistic participants to be similarly confident to controls (but less accurate) in FER tasks that also involved unmasked faces. This apparent discrepancy appears to reflect the Dunning-Kruger effect, whereby a person can either over-estimate or under-estimate their ability at a task [64]. The bias that is expressed by those who trend towards autism seems to depend on the aspect of the task that is emphasised. When performance is emphasised-as in our study and that of Bekele et al. [62]-participants underestimate their competence (i.e., their confidence is below average). And when confidence is emphasised, as in Sawyer et al. [63] who instructed participants to choose the level that balanced speed and accuracy, participants overestimate their competence (i.e., their performance is below average).
As in the first two experiments, high scorers in Experiment 3 were less confident in identifying anger in masked female faces than in masked male faces, and high scores rated happiness in masked male faces as more intense than in masked female faces. High scorers were also less confident than low scorers in categorizing happiness in both male and female masked faces. Similar to Experiment 1, high scorers rated fear as more intense in masked male faces than in masked female faces. Thus, while the introduction of masks did not hinder FER accuracy significantly more for high scorers on the AQ-10, high scorers' confidence in emotion recognition and perception of emotional intensity for certain emotions were impacted by masks more than low scorers. Confidence was particularly reduced for high scorers when viewing masked faces labeled as disgust, happy, or neutral, whereas female masked faces labeled as happiness, and disgust were rated as significantly less intense.
In summary, our results support previous research showing that wearing facial masks decreases both facial expression recognition [18,21,22], and confidence in expression identification [22]. Contrary to previous research that found no effects of facial masks on recognition of fear [22], neutrality [18,21,22] or sadness [21] our studies showed that facial masks disrupt recognition of all investigated emotions. We also had the novel finding that perception of expression intensity of anger, disgust, fear, happiness, neutrality, and sadness were reduced by masks. Our novel finding that high AQ-10 scorers were lower in FER accuracy, FER confidence, and emotion perception relative to low AQ-10 scorers when viewing masked faces adds to the extant COVID-19 autism-related research, as well as to the broader FER and ASD research. These novel findings are consistent with previous research findings that people with autistic traits have greater difficulty than NTs in FER [30,65,66]. Our results suggest that the introduction of mask wearing in the current pandemic climate will reduce confidence in people with autistic traits in FER, particularly when attempting to discern fearful, neutral, or happy faces. The current investigation also extends the previous research by showing that masks affect all expressions in terms of recognition, confidence in identification, and intensity perception. Moreover, this research contributes to the literature by confirming these effects in individuals with autistic traits and comparing them with neurotypicals.
Possible limitations in the current study include that reaction times were not recorded. While there were no significant differences between the two AQ-10 groups in FER accuracy for masked faces, it is not possible to determine whether or not high scorers took longer to label facial expressions. As autistic individuals can take longer to process faces [67,68], future studies should examine potential differences in response times between individuals with autistic traits and neurotypicals.
Another limitation was that our participants were predominantly female for all three studies. Had there been a better balance between male and female participants, we could have explored differences between males and females. Additionally, the images used in the study exhibited emotions at their peak intensity, which is not always the case during natural social interactions. Given that high scorers reported the expressions in the stimuli as less intense than low scorers did, their confidence and accuracy might be further reduced when interacting with others in day-to-day social situations when expressions might not be delivered at peak intensity, with or without masks. Moreover, all three experiments were conducted online, and while online experiments are shown to be reliable and have external validity [69], we suggest that future research investigate the effect of facial masks in FER by testing individuals who score high and low on the AQ-10 in person. Also, as dynamic stimuli can facilitate emotion perception [42,43], we suggest it would be instructive to examine the effect of facial masks on FER when NTs and autistic individuals are presented with dynamic versus static images.
In conclusion, across three experiments in which we tested individuals from different populations and with different levels of autistic traits, we found that facial masks have a negative effect on facial expression recognition. Wearing facial masks also reduces the intensity of the emotion that is being perceived and observers' confidence in their ability to correctly identify the emotion, particularly so for those with autistic traits.