Certain facial configurations are believed to be associated with distinct affective meanings (i.e. basic facial expressions), and such associations are common across cultures (i.e. universality of facial expressions). However, recently, many studies suggest that various types of contextual information, rather than facial configuration itself, are important factor for facial emotion perception.
To examine systematically how contextual information influences individuals’ facial emotion perception, the present study estimated direct observers’ perceptual thresholds for detecting negative facial expressions via a forced-choice psychophysical procedure using faces embedded in various emotional contexts. We additionally measured the individual differences in affective information-processing tendency (BIS/BAS) as a possible factor that may determine the extent to which contextual information on facial emotion perception is used. It was found that contextual information influenced observers' perceptual thresholds for facial emotion. Importantly, individuals’ affective-information tendencies modulated the extent to which they incorporated context information into their facial emotion perceptions.
The findings of this study suggest that facial emotion perception not only depends on facial configuration, but the context in which the face appears as well. This contextual influence appeared differently with individual’s characteristics of information processing. In summary, we conclude that individual character traits, as well as facial configuration and the context in which a face appears, need to be taken into consideration regarding facial emotional perception.
Citation: Lee T-H, Choi J-S, Cho YS (2012) Context Modulation of Facial Emotion Perception Differed by Individual Difference. PLoS ONE 7(3): e32987. https://doi.org/10.1371/journal.pone.0032987
Editor: Alessio Avenanti, University of Bologna, Italy
Received: August 15, 2011; Accepted: February 3, 2012; Published: March 14, 2012
Copyright: © 2012 Lee et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by a Korea Research Foundation Grant, funded by the Korean Government (MOEHRD) (KRF-2008-321-H00008). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Imagine a poker player trying to guess an opponent’s hand. The player "reads" some ambiguous information about the opponent's hand in the opponent's facial expression. The perception of facial cues largely depends on the perceiver. The hand (e.g., busted hand or full house) and the character (e.g., nervous cat or bold lion) possessed by the player can greatly influence the player's interpretation of the opponent's subtle facial expressions. Despite general acceptance that context and individual factors influence people's abilities to perceive others' emotions via their expressions, most face perception studies have focused on the universality of the expressions. These studies have provided a strong theoretical foundation for a robust connection between a certain set of facial configurations and so-called “basic emotions".
Recent studies have challenged this view –. Research has shown that various external surrounding factors, such as race , body posture , , verbal explanation of situation , , other emotional faces , and emotional scene – modulate the emotion perception of facial configuration. For example, in Aviezer et al.’s study (2008), which asked observers to rate the valence and arousal of a face embedded in varying contexts (e.g., body posture), observers recognized the same facial expressions as different emotions depending on the face's contextual image. These studies have shown that context information plays a role at an early, visual stage involved in face encoding processing , , at a late, processing stage involved in interpretation and perceptual decision , or at both the stages , , . Together, these earlier results highlight the importance of surrounding circumstances, namely, context information, for perceiving emotions from facial configuration.
However, not all studies necessarily showed that context information is a predominant factor in facial emotion perception , . For example, in Nakamura et al.’s study (1990), when participants rated the emotional state of a facial configuration presented simultaneously with situational information, facial configuration itself had a larger effect on the emotion rating than situational information. The authors concluded that the recognition of facial emotion is primarily determined by facial configuration.
It has been found that information processing of emotional stimuli varies as a function of individual differences (see  for a review). For example, Carver and White demonstrated that individual’s behavioral tendencies of approach and avoidance are closely related to the tendency to process emotional information . Therefore, individual differences could have affected the extent to which observers used emotional context information in facial emotion perception in the previous studies. In light of this, the present study focuses on each individual’s Behavioral Inhibition System (BIS), which is conceptually linked to approach motivation, and Behavioral Approach System (BAS; , ), which is linked to avoidance motivation, because BIS/BAS levels are associated, respectively, with the processing of negative and positive information , . In particular, both the BIS  and BAS  modulate the activity of the amygdala, a central component in emotion processing. Thus, the present study examines the influences of individual’s BIS/BAS on facial emotion perception with emotionally different contextual information.
The present study conducted a face-emotion perceptual task to test the roles of both context information and tendency to attend to emotional information in facial emotion perception. There are several ways to define the concept of context. For example, Chun (2000) defined it as elements and the way those elements are configured to form a complex set of stimuli . However, in the present study, context is defined as a functionally relevant set of stimuli that can modulate a response, but not directly drive it, often by influencing how information is sampled from other stimuli that are more causally linked to the target response , . For this task, a range of morphed faces, with varying emotional intensities, was overlaid on one of three emotional contexts: negative, positive, or neutral. Observers were asked to indicate whether the face in each contextual image was fearful or neutral and observers’ behavioral performance was characterized via psychometric curves. In this manner, we directly estimated each observer’s perceptual thresholds for identifying a face as fearful in different contexts. To assess the modulatory effect of individual differences, we measured the observers’ affective information-processing tendency regarding BIS/BAS.
The study was approved by the Ethics Committee of the Korea University. Fifteen observers with corrected-to-normal vision volunteered for this study. We obtained written informed consent from all observers.
Stimuli and apparatus
Face stimuli (fearful and neutral) were taken from the Korea University Facial Expression Collection . To vary emotional intensity parametrically, we morphed the stimulus faces from neutral (0%) to fearful (100%) in 10% increments (Figure 1A). We employed 11 separate increments during the two-forced choice task, after equalizing the different face stimuli for luminosity. Each face was enclosed in a rectangular frame (3.0°×3.3°) excluding most of the hair and non-facial contours. We selected the context images from the International Affective Picture System (IAPS; ), employing 25 negative, 25 neutral, and 25 positive images (21°×15°) as negative, neutral, and positive context stimuli (see Table. S1). To obtain the observers’ unbiased responses, we used additional Fourier transformed images to create a baseline. Superlab 4.0 with a RB-730 response pad (Cedrus, CA) was used to program the experiment. Stimuli were presented on a personal computer's 21-in. display screen, at a viewing distance of approximately 60 cm.
Observers were told to place their index fingers on the response keys. The experiment consisted of one practice session of 10 trials, followed by four test sessions, one for each context: negative, neutral, positive, and baseline. Each session consisted of two blocks of 275-trials. Thus, each observer performed 2,250 trials (25-presentations of a face stimulus at each intensity×11 face intensities×2 blocks×4 contexts). Observers responded to the baseline condition first. Following the baseline session, the order of the other contexts was counterbalanced across observers. Rest breaks were given as necessary to avoid observer fatigue.
Each trial began with a 500 ms fixation cross (0.95°×0.95°), followed by 50 ms of blank screen. Observers were asked to stare at the fixation. Next, a context stimulus was presented first. After 400 ms, the stimulus face was presented centrally, on the context, for 100 ms. The context stimulus remained visible until the face stimulus terminated. At stimulus offset, a mask, which was a white-black checkerboard rectangle (21°×15°) replaced the stimuli. After 100 ms, the response screen appeared until the observer made a response (Figure 1B). Observers indicated whether the stimulus was a fearful or neutral face by pressing one of two response keys. The inter-trial interval was 1,000 ms.
To obtain psychometric curves for the different contexts, we fitted the behavioral data for each context to the curves via the Naka-Rushton contrast response model (see Equation 1)(1)
(Equation 1. Naka-Rushton Response Function)
–where Response represents the percentage of “fearful” responses, C is the stimulus intensity level (contrast), C50 is the contrast at half the saturating response (perceptual threshold), n is the exponent that determines the steepness of the function (slope), Rmax is the maximum response relative to the baseline (asymptote), and M is the response at the lowest intensity level. To fit the data to the model, we constrained the parameters C50, n, and Rmax, based on each individual’s baseline context results. Curve fitting was performed with a maximum likelihood criterion. To best fit the data to the model, we fit the baseline condition data first with allowing each parameter (i.e. C50, Rmax, and n) to vary freely. Next, the curve fit to each context condition across observers was constrained fit bounds by first obtaining estimates from the baseline condition (final C50 ranging from .2894 to .5863; M from 0 to .1168; n from .0572 to 14.84; Rmax from .7269 to 1.0).
To measure each observer’s information-processing characteristics, we used a BIS/BAS questionnaire that included one BIS scale and three BAS scales: Drive, Fun Seeking, and Reward. Because the BIS scale and the BAS reward scale depend upon one’s information processing tendencies toward external events, information, and the world , , we focused on BIS and BAS reward scales in the present study. To examine the relationship between BIS/BAS tendencies and contextual influences, context effect scores were calculated for both the negative and positive contexts, by subtracting the threshold value for the neutral context from that of the appropriate emotional context. A context effect score<0 indicated a decrease in the threshold.
Context Modulation of Perceptual Threshold
For average data, the estimated perceptual threshold (C50) function for fearful faces in a negative context shifted consistently to the left relative to its function in neutral context, when the expression intensity was ambiguous (30–60% intensities). The C50 function in the positive context shifted consistently to the right relative to that obtained in neutral context, even though modulation was not as great as it was in negative context (Figure 2A and B). Observers perceived more fearful faces in negative context (C50 = .3985) than in neutral context (C50 = .4312). Conversely, positive context led to a consistent increase in threshold (C50 = .4449) relative to neutral context.
(A) Psychometric functions fitted to the average response data. (B) The effects of emotional context on C50, for each observer. Points falling on the gray line represent unity, where there is no difference between emotional and neutral contexts. Points under the line indicate increased thresholds, and points over the line indicate decreased thresholds. (C) Scatter plot of the relationship between the observer’s context effect score and their informational tendency (BIS/BAS) with regard to facial emotion perception. A context effect score below zero indicates a decrease in the threshold value, and a context effect score above zero indicates an increase in the threshold value.
To further explore the data, the individual psychometric functions were estimated (mean R2 = .97, range = .82–99). A statistical test revealed a significant difference in the C50 parameter between the context conditions, F(2, 28) = 7.83, p<.005, η2 = 36. A subsequent comparisons (LSD) showed that the threshold was significantly lower in negative context (C50 = .4154) than neutral context (C50 = .4360), p<.05, but it was marginally higher in positive context (C50 = .4548) than neutral context, p<.05. Also, there was a significant difference between negative and positive context, p<.01. The percentage of reported “fearful” responses in the intensity of the stimulus was not modulated by contextual information when expression intensity was low (0–20%) or high (70–100%), indicating that observers’ responses were based on the facial configuration rather than context image.
Relationship between BIS/BAS and Context Effect
The individual’s information processing tendencies modulated the context effect obtained (Figure 2C). To examine the relative contributions of BIS and BAS tendencies to the context effect, a stepwise multiple regression analysis was conducted on the entire dataset, entering the BIS and BAS values as independent variables and the context effect score as the dependent variable. In the negative context, only BIS tendency contributed significantly to the context effect score (β = –.545, p<.05), whereas BAS tendency did not (β = −.055, p = .82). Conversely, in the positive context, BAS tendency contributed significantly to the score (β = .689, p<.005), while BIS did not (β = .367, p = .068).
Other people’s affective facial expression should be interpreted in terms of the context in which it occurs for people to adequately behave in social situation. The present study investigated how contextual information influences individuals’ facial emotion perception, by directly estimating each observers’ thresholds for perceiving facial expressions. The findings reported here support the idea that people generally tend to use contextual information when discerning the emotion of a face. The observer’s perceptual threshold (i.e. C50) for a fearful face changed as a function of contexts (i.e., shifting rightward), supporting the previous studies showing that context information could modulate facial emotion perception –. Surprisingly, individuals’ BIS/BAS levels correlated with the extent to which context information influenced facial emotion perception. When facial emotion was ambiguous, the negative contextual images strongly influenced the facial-expression perception of the observers with high BIS tendencies (r = –.545, p<.05), whereas positive images influenced the perception of the observers with high BAS tendencies (r = .689, p<.005).
Growing body of evidence indicates that facial emotion perception is modulated by the context. For example, Mobbs et al.,(2007) reported that both negative and positive contexts yielded a significant difference for face perceptions in behavioral rating and brain activities. Similarly, using a conditioning procedure, Lim and Pessoa (2008) have shown that the perceptual threshold was significantly lower for a face in a color background that has been previously paired with an aversive stimulus . In comparison, our study, employing psychophysical measurement, revealed a bidirectional context modulation of face perception for fearful face (Figure 2A). The observers were more likely to perceive ambiguous faces (e.g., 30 – 60% intensities) as fearful in the negative context than they were in the neutral context. On the other hand, they were less likely to perceive the faces as fearful in the positive context. Taken together, these studies indicate that perceptual thresholds for emotional faces are modulated by functional relevance.
What are the mechanisms by which the context information modulated facial emotion perception? One possible explanation for such a shift in the C50 is that visual processing at the stage of facial encoding was modulated by contextual information. It has been known that the amygdala activity that emotional information triggers leads to enhanced visual processing (see also , ) through the neural connections between the visual area (e.g., V1) and the amygdaloid regions , . Indeed, it has been found that contextual information leads to amygdala activation in face stimulus processing , . For example, Mobbs et al. (2007) observed an interaction between the amygdala and the visual areas, including face processing regions such as fusiform gyrus (FFG) and superior temporal sulcus (STS), when they presented observers with faces in contextual scenes. The present findings regarding individual difference in BIS/BAS tendencies also support the above mechanism considering the previous researches showing relationships between amygdala activation and the level of BIS/BAS , . That is, the level of BIS/BAS determines the degree of amygdala activation for the contextual emotional information, resulting in the degree of changes in the visual processing of the faces. Consequently, observers altered the perceived emotion from the face in accordance with their degree of sensitivity changes.
However, because amygdala activation is not just caused by negative information , , the positive context should have resulted in a decrease in the C50 function, too. The current findings-an increase in C50 function (i.e., a rightward shift) in the positive context-suggest that context information had an influence on the late-processing stages of interpretation and perceptual decisions, as well as at the early perceptual-processing stages. Indeed, Righart and de Gelder (2008b) found that observers’ responses to a facial expression they saw in an emotionally incongruent scene tended to be biased toward the contextual emotion, not to the face emotion, indicating that context information directly modifies observers’ interpretations of what a facial expression means in the late stages. Also, when perceiving the emotions of faces superimposed on emotional contexts, neural activation occurs in ventral prefrontal cortex (vPFC; ), which plays a key role in top-down processing, including inference, expectation, and decision-making –. That is, the present findings are consistent with the idea that the degree to which top-down processing contributes facial emotion perception depends on the observers’ subjective relevance or salience of emotional context information , , .
Although individual differences was found to be an important factor modulating such contextual influence on facial expression perception in the present study, other factors are worthy of consideration when researchers investigate contextual influences on face emotion perception. One such factor could be the observers' cultural differences. A recent study demonstrated that Japanese and American participants differed in utilizing contextual information for emotion perception ; the East Asians tended to perceive objects within their given context, whereas the Westerners focused more on each focal object, independently of its context . Moreover, the effect of context on facial emotion perception also varies as a function of the observer’s age and level of stress .
As in previous studies –, , the present results challenged the major view of facial expression perception indicating that facial configuration is interpreted in terms of its surrounding context information to perceive facial emotion, rather than it is just recognized in terms of its features. More important, the way how contextual information affects the facial expression perception depends on the observer’s individual tendency to process positive or negative information. These findings shed light on the factors that contribute to emotion perception for the face in the real world; perceptual cue, context information, and observer’s tendency to use context information. Thus, consideration of these factors is necessary to understand how to perceive emotion in a social setting.
Average Normative Valence and Arousal (Standard Deviations) and Picture Identification Numbers from the International Affective Picture System for the Negative, Neutral and Positive Pictures Used (valence scale ranging from 1: negative – 9: positive; arousal from 1: calm – 9: aroused).
We thank Seung-Lark Lim, Hackjin Kim, and Jeong-Seog Song for conceptual feedback and technical advice about psychophysical function.
Conceived and designed the experiments: THL YSC. Performed the experiments: THL. Analyzed the data: THL YSC. Contributed reagents/materials/analysis tools: THL YSC. Wrote the paper: THL YSC JSC. Provided critical revisions important for intellectual content: JSC YSC Analyzed and interpreted the data: THL JSC YSC.
- 1. Ekman P (1993) Facial expression and emotion. Am Psychol 48: 384–392.
- 2. Aviezer H, Hassin RR, Bentin S, Trope Y (2008) Putting facial expressions back in context. In: Ambady InN, (Eds.) JSkowronsky, editors. First impression. Guilford press: New York. pp. 255–286.
- 3. Barrett LF, Lindquist KA, Gendron M (2007) Language as context for the perception of emotion. Trends in Cognitive Sciences 11: 327–332.
- 4. de Gelder B, Meeren HK, Righart R, van den Stock J, van de Riet WA, et al. (2006) Beyond the face: exploring rapid influences of context on face processing. Prog Brain Res 155: 37–48.
- 5. Hugenberg K (2005) Social categorization and the perception of facial affect: target race moderates the response latency advantage for happy faces. Emotion 5: 267–276.
- 6. Aviezer H, Hassin RR, Ryan J, Grady C, Susskind J, et al. (2008) Angry, disgusted, or afraid? Studies on the malleability of emotion perception. Psychol Sci 19: 724–732.
- 7. Meeren HK, Heijnsbergen CC, de Gelder B (2005) Rapid perceptual integration of facial expression and emotional body language. Proceedings of the National Academy of Science, USA 102: 16518–16523.
- 8. Carroll JM, Rusell JA (1996) Do facial expressions signal specific emotions? Judging emotions from the face in the context. Journal of Personality and Social Psychology 70: 205–218.
- 9. Kim H, Somerville LH, Johnstone T, Polis S, Alexander AL, et al. (2004) Contextual modulation of amygdala responsivity to surprised faces. J Cogn Neurosci 16: 1730–1745.
- 10. Masuda T, Ellsworth PC, Mesquita B, Leu J, Tanida S, et al. (2008) Placing the face in context: cultural differences in the perception of facial emotion. J Pers Soc Psychol 94: 365–381.
- 11. Mobbs D, Weiskopf N, Lau HC, Featherston E, Dolan RJ, et al. (2007) The kuleshov effect: the influence of contextual framing on emotional attributions. Soc Cogn Affec Neurosci 1: 95–106.
- 12. Righart R, de Gelder B (2008) Rapid influence of emotional scenes on encoding of facial expressions: an ERP study. Soc Cogn Affect Neurosci 3: 270–278.
- 13. Righart R, de Gelder B (2008) Recognition of facial expressions is influenced by emotional scene gist. Cognitive, Affective, & Behavioral Neuroscience 8: 264–272.
- 14. Righart R, de Gelder B (2006) Context influences early perceptual analysis of faces–an electrophysiological study. Cereb Cortex 16: 1249–1257.
- 15. Clark BM, Siddle DAT, Bond NW (1992) Effects of social anxiety and facial expression on habituation of the electrodermal orienting response. Biological Psychology 33: 211–223.
- 16. Fernandez-Dols JM, Sierra B, Ruiz-Belda MA (1993) On the clarity of expressive and contextual information in the recognition of emotions: A methodological critique. European Journal of Social Psychology 23: 195–202.
- 17. Nakamura M, Buck R, Kenny D (1990) Relative contributions of expressive behavior and contextual information to the judgment of the emotional state of another. Journal of Personality and Social Psychology 59: 1032–1039.
- 18. Humphreys MS, Revelle W (1984) Personality, motivation, and performance: a theory of the relationship between individual differences and information processing. Psychol Rev 91: 153–184.
- 19. Carver CS, White TL (1994) Behavioral inhibition, Behavioral activation, and affective responses to impending reward and punishment: the BIS?BAS scales. Journal of Personality and Social Psychology 67: 319–333.
- 20. Gray JA (1970) The psychophysiological basis of introversion-extraversion. Behav Res Ther 8: 249–266.
- 21. Noguchi K, Gohm CL, Dalsky DJ (2006) Cognitive tendencies of focusing on positive and negative information. Journal of Research in Personality 40: 891–910.
- 22. Rusting CL, Larsen RJ (1998) Personality and Cognitive Processing of Affective Information. Pers Soc Psychol Bull 24: 200–213.
- 23. Reuter M, Stark R, Hennig J, Walter B, Kirsch P, et al. (2004) Personality and emotion: test of Gray's personality theory by means of an fMRI study. Behav Neurosci 118: 462–469.
- 24. Beaver JD, Lawrence AD, Passamonti L, Calder AJ (2008) Appetitive motivation predicts the neural response to facial signals of aggression. J Neurosci 28: 2719–2725.
- 25. Chun MM (2000) Contextual cueing of visual attention. Trends Cogn Sci 4: 170–178.
- 26. Barrett LF, Kensinger EA (2010) Context is routinely encoded during emotion perception. Psychol Sci 21: 595–599.
- 27. Barrett LF, Mesquita B, Gendron M (2011) Context in Emotion Perception. Current Directions in Psychological Science 20: 286–290.
- 28. Lee T-H, Lee K, Lee K-Y, Choi J-S, Kim HT (2006) The Korea University Facial Expression Collection: KUFEC. Seoul, South Korea: Lab of Behavioral Neuroscience Dept of Psychology, Korea University.
- 29. Lang PJ, Bradely MM, Cuthbert BN (1999) International affective picture system (IAPS) : Instruction manual and affective ratings. Gainesville: University of Florida, Center for Reseach in Psychophysiology.
- 30. Sclar G, Maunsell JH, Lennie P (1990) Coding of image contrast in central visual pathways of the macaque monkey. Vision Res 30: 1–10.
- 31. Smits Dirk JM, Boeck PD (2006) From BIS/BAS to the big five. European journal of personality 20: 255–270.
- 32. Lim SL, Pessoa L (2008) Affective learning increases sensitivity to graded emotional faces. Emotion 8: 96–103.
- 33. Amaral DG, Behniea H, Kelly JL (2003) Topographic organization of projections from the amygdala to the visual cortex in the macaque monkey. Neuroscience 118: 1099–1120.
- 34. Padmala S, Pessoa L (2008) Affective Learning Enhances Visual Detection and Responses in Primary Visual Cortex. J Neurosci 28: 6202–6210.
- 35. Phelps EA, Ling S, Carrasco M (2006) Emotion facilitates perception and potentiates the perceptual benefits of attention. Psychol Sci 17: 292–299.
- 36. Davis M, Whalen PJ (2001) The amygdala: vigilance and emotion. Mol Psychiatry 6: 13–34.
- 37. LeDoux JE (2000) Emotion circuits in the brain. Annu Rev Neurosci 23: 155–184.
- 38. Fitzgerald DA, Angstadt M, Jelsone LM, Nathan PJ, Phan KL (2006) Beyond threat: Amygdala reactivity across multiple expressions of facial affect. NeuroImage 30: 1441–1448.
- 39. Somerville LH, Kim H, Johnstone T, Alexander AL, Whalen PJ (2004) Human amygdala responses during presentation of happy and neutral faces: correlations with state anxiety. Biological Psychiatry 55: 897–903.
- 40. Clark L, Cools R, Robbins TW (2004) The neuropsychology of ventral prefrontal cortex: Decision-making and reversal learning. Brain and Cognition 55: 41–53.
- 41. Fenske MJ, Aminoff E, Gronau N, Bar M (2006) Top-down facilitation of visual object recognition: object-based and context-based contributions. Prog Brain Res 155: 3–21.
- 42. Frith C, Dolan RJ (1997) Brain mechanisms associated with top-down processes in perception. Philos Trans R Soc Lond B Biol Sci 352: 1221–1230.
- 43. Vrana SR, Gross D (2004) Reactions to facial expressions: effects of social context and speech anxiety on responses to neutral, anger, and joy expressions. Biological Psychology 66: 63–78.
- 44. Ko SG, Lee TH, Yoon HY, Kwon JH, Mather M (2011) How does context affect assessments of facial emotion? The role of culture and age. Psychol Aging 26: 48–59.
- 45. Nisbett RE, Masuda T (2003) Culture and point of view. Proc Natl Acad Sci U S A 100: 11163–11170.