Face-to-face social behaviour is difficult to explain, leading some researchers to call it the “dark matter” of psychology/neuroscience . We apply an idea from neuroeconomics to this problem, suggesting that how people subjectively value facial expressions should predict usage differences during unconstrained interaction. Specifically, we ask whether the subjective value of smiles is malleable as a consequence of immediate social experience and how this relates to smiling during face-to-face interactions. We measured the value of a smile in monetary terms and found that increases in people’s social neediness caused devaluation of polite smiles but no changes in how they valued genuine smiles. This result predicts that participants induced to feel high levels of social need should be less responsive to their social partners’ polite smiles in a subsequent unconstrained social interaction. As expected, high social-need participants returned fewer polite smiles when interacting with a partner, leading to poor interaction outcomes. Genuine smile reciprocity remained unchanged. Findings show that social states influence real-world interactions by changing social-cue valuation, highlighting a potential mechanism for understanding the moment-to-moment control of social behaviour and how behaviour changes based on people’s subjective evaluations of the social environment.
Citation: Heerey EA, Gilder TSE (2019) The subjective value of a smile alters social behaviour. PLoS ONE 14(12): e0225284. https://doi.org/10.1371/journal.pone.0225284
Editor: Marina A. Pavlova, Univdersity Hospital of TübingenUniversitatsklinikum Tubingen, GERMANY
Received: June 18, 2019; Accepted: October 31, 2019; Published: December 2, 2019
Copyright: © 2019 Heerey, Gilder. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: De-identified raw and summary data (Experiment 1) and coded interaction data (Experiment 2) may be obtained from https://osf.io/mxt63/ (interaction videos cannot be shared, as participants have not consented to the release of image/voice files to individuals outside the research team).
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Genuine smiles of pleasure are rewarding social cues that cause positive feelings in receivers and lead people to anticipate positive social outcomes [2–4]. When provided as reinforcers for social decisions, genuine smiles enhance learning and activate reward circuitry in the brain [5, 6]. Polite smiles occur in the absence of positive emotion, when politeness rules necessitate a smile . Although not typically considered rewarding [3, 8], polite smiles are nonetheless important social tokens . Indeed, both genuine and polite smiles are highly likely to be reciprocated in face-to-face social interactions and good smile reciprocity enhances liking for a social partner [10–12]. Thus, these smiles likely play an important role in social decision-making.
Examined in the context of economic choices, smiles are also important social cues. Findings suggest that smiles alter decision-making such that participants are willing to invest more money with smiling versus non-smiling partners in trust games, even when returns are lower [13, 14]. Smiles also add value to choices in simple decision-making tasks, meaning that participants are willing to forego the chance to earn monetary rewards in favour of the chance to see genuine smiles . Thus, like financial incentives, smiles may shape the decisions people make by altering the “utility,” or subjective value, of the actions with which they are associated. If fluctuations in smile value change moment-to-moment responses to these cues in social interactions, such a result would highlight one mechanism underpinning real-world social behaviour. Here we ask whether the subjective value of a smile depends on social states, just as physical states (e.g., hunger or satiety) change the subjective value of associated stimuli (e.g., food [16, 17]). We then ask whether alterations in smile value determine responses to smiles in face-to-face interaction. An important dimension of people’s social states is the degree to which they feel accepted versus rejected by their peers [18–21]. The experience of social rejection, leading to a state of enhanced social need, dramatically influences immediate emotions and social behaviours [22–26] and, in cases of chronic rejection, predicts poor long-term outcomes [27, 28]. We anticipate that altering social need state by inducing immediate feelings of social rejection will influence the subjective values of different types of smiles. For example, after experiencing a social rejection manipulation, individuals show better ability to discriminate genuine smiles from polite smiles, greater visual attention to genuine smiles, and express greater desire to work with genuinely versus politely smiling partners [29–31].
Such findings have led researchers to hypothesize that genuine smiles become more valuable under states of high social need because they predict positive social outcomes, thereby appeasing rejected individuals’ need for social belonging [18, 29, 32, 33]. However, whether smile-value is indeed malleable remains unknown, as this has never been directly tested. Moreover, because genuine and polite smiles are typically compared to each other, rather than to a neutral/control condition (e.g., [29, 34]), it remains unclear whether rejection-induced changes in social state alter the value of genuine smiles, polite smiles, or both. This is an important question because it speaks directly to a possible mechanism by which social state changes may alter face-to-face social behaviour and subsequent social outcomes after enduring a social rejection [18, 19, 21, 22, 35, 36].
If rejection experiences cause changes in smile valuation that in turn affect social behaviour, these changes are likely to have major downstream consequences for interaction. For example, previous research shows that in natural interactions participants reciprocate 70% to 80% of their partners’ smiles. More importantly however, they match their partners’ smile types such that a partner’s genuine smiles lead people to return genuine smiles and polite smiles lead to polite smile returns [10, 37]. Evidence additionally suggests that the value of a smile shapes some aspects of this reciprocity, such as its timing . Thus, changes in smile value as a result of social state fluctuations may change how participants view and experience partners’ social behaviour.
We note that the purpose of this research is not to make a direct contribution to the social rejection literature. Rather, the major point of this work is to contribute to the literature on how changes in people’s subjective affective experiences drive their face-to-face social behaviour. We consequently propose the hypothesis that social state changes will alter the subjective utility of smiles, and thereby change how people respond to their partners’ smiles. This work therefore offers the first direct test of the longstanding prediction that rejection-induced changes in social need enhance the value of genuine smiles and we include the critical neutral condition, allowing us to examine smile-value changes for genuine and polite smiles separately.
To investigate the effects of changes in how people respond to smiles, we adopt a neuroeconomic approach to the problem of quantifying smile utility. Specifically, we adapted a task commonly used in studies of economic utility (e.g., [38, 39, 40]) that allows us to measure the value of genuine and polite smiles in monetary terms [15, 41]. In utility tasks, participants must choose between pairs of stimuli that vary in value on one or more dimensions. One of the dimensions is often “subjective,” for example, how much a particular person desires merchandise from a set of companies (e.g., Apple, Samsung). The other dimension is typically quantifiable using a standard currency, for example, how much the person is willing to pay to obtain the branded merchandise (e.g., [42, 43]).
In order to represent subjective value in monetary terms, participants make choices amongst pairs of stimuli. Because all possible pairings of stimuli are presented and participants must always select one stimulus per trial, their selections across this decision-space provide a measure of their relative preferences for each stimulus in the set. It is possible to precisely quantify subjective preferences by determining how much additional currency people are willing to pay in order to receive a desired, relative to a less desired stimulus. This method is advantageous to simple preference reporting, because it enables precise quantification of relative preferences for one stimulus dimension (e.g., brand-name) standardized in terms of the other dimension (e.g., cost).
Our version of this task uses a set of faces that provide monetary reinforcement with either higher or lower frequency and give different types of social feedback believed to vary in subjective value [38, 40]. We use a two-step procedure in which participants experience and learn both the monetary and social values of our stimuli, followed by a test phase in which they must choose between stimuli presented in pairs. Based on the existing literature, we predict that rejection-induced increases in social need will enhance the subjective value of genuine smiles. That is, participants who experience a social rejection should be willing to forgo a greater amount of money for the chance to see genuine smiles relative to participants who have not undergone a social need induction.
We begin by quantifying how social state (e.g., feeling rejected versus accepted by a future social partner) influences the subjective value of smiles.
Ninety psychology undergraduates (62 female; mean age: 20.47 years, SD: 4.56) participated in “a study of personality characteristics and first impressions” in exchange for partial course credit and a small performance-based monetary bonus. We determined the sample size before the start of data collection, based on a power analysis. We estimated effect sizes based on effect-size data reported in two previously published experiments examining responses to genuine and non-genuine smiles [29, 34]. A G*Power  analysis (effect size f = .591, α = .05, 1-β = .95) suggested groups of 16 participants each. However, we oversampled to obtain 30 participants per condition (after exclusions). Potential participants were pre-screened to rule out the presence of social anxiety (defined as a score >60 on the Interaction Anxiousness Scale, ), which may interfere with the processing of social stimuli . No person scoring more than 60 on the Interaction Anxiousness Scale was invited to participate in the present study. Participants provided written consent and the Bangor University School of Psychology Ethics Committee approved the experiment and all study procedures (Likewise for Experiment 2 below). We excluded data from four additional participants because they reported suspicion about the experiment’s manipulation.
Participants arrived for the experiment in pairs to give the impression that they would be partners for a social interaction. In reality, they did not have a face-to-face interaction but instead completed a computer task measuring the utility of genuine and polite smiles, relative to neutral faces. Upon arrival, the experimenter seated participants at computers in separate rooms where they completed a series of questionnaires measuring aspects of personality (not analyzed), along with the Positive and Negative Affect Schedule (PANAS ) to measure mood. We measured social need state by inserting six rejection-related words into the PANAS at random points (slighted, misunderstood, rejected, liked [reversed], respected [reversed], supported [reversed]; Cronbach’s α = .707). After completing these measures, participants viewed a personality profile, ostensibly of the partner. In reality, however, all participants viewed the same profile.
Based upon this personality profile, participants responded to a series of statements about how much they looked forward to meeting the partner (e.g., “I’m looking forward to meeting my partner”) and then received false feedback about how much the partner looked forward to meeting them (for similar procedures see [48, 49]). The computer then randomly assigned participants to one of three feedback conditions (see Supporting Information Pilot Results and S1 Fig for full text and validation information). These were 1) acceptance feedback (low social need; “Your partner is looking forward to meeting you.”; n = 30); 2) social rejection feedback (high social need plus high negative affect; “Your partner is not looking forward to meeting you.”; n = 30); and 3) negative, non-rejection feedback (high negative affect only; “Your partner is not looking forward to meeting anyone.”; n = 30). Importantly, this third condition allowed us to determine whether changes in smile utility related specifically to social need state versus negative emotion more generally. An independent set of pilot participants assessed the manipulation to ensure that it was both believable and caused the desired outcomes in receivers (see S1 Fig for pilot results). Finally, because the computer assigned feedback to each participant, the experimenter was blind to participant condition until the conclusion of the experiment.
After viewing the feedback, participants completed a second PANAS (including social-need items), a computer-based smile-valuation task that allowed us to measure the utility of smiles in monetary terms (see below), and a short task in which they distinguished genuine smiles from polite smiles. Following the tasks, the experimenter probed participants for suspicion about the manipulation, explained the manipulation and fully debriefed them, paid them based on their earnings in the smile-valuation task and finally gave them the opportunity to provide fully informed consent using a secret “sealed envelope” procedure, which ensured that the experimenter was unaware of their final response until after participants had been dismissed. No participant declined consent.
This task has both exposure and test phases. The purpose of the exposure phase was to allow participants to become familiar with a set of faces that varied on two reward dimensions (monetary value and social value). This phase allowed us to ensure that participants experienced the exact same reinforcement frequencies and types in each monetary and social value condition. During this phase, participants experienced a set of six opponents, each identified by a photograph of a face.
On each trial of the exposure phase, participants viewed one opponent, neutrally posed, in the center of the screen and attempted to win money by choosing the same side of a virtual coin as the opponent (Fig 1a). To indicate a match (each worth .02GBP), the opponent smiled genuinely (involving zygomaticus major and orbicularis oculi); smiled politely (zygomaticus major only); or remained in the neutral pose with text superimposed on the screen indicating the win (for a full description of the smile stimuli and their psychometric properties, see ). On non-win trials, superimposed text appeared indicating a non-win for the neutral opponents whereas the other opponents frowned.
In the exposure phase, a) Participants attempted to choose the same side of the coin as a computerized player (indicated by a photograph) using a key press. Players indicated match (worth .02GBP) and non-match outcomes (worth 0GBP) with either facial or text displays. There were three high-value faces (providing win feedback on 80% of trials, regardless of participants’ choices) and three low-value faces (60% win feedback). One high-value and one low-value face provided match feedback with genuine smiles; likewise for polite smiles and text feedback. b) On each trial in the test phase, participants chose the “most valuable” player from amongst pairs of players, prior to playing out the round. All possible pairings were represented. These choices served as the dependent variable in the task. This task is identical to the task described in . Note that the images participants viewed during that task were photographs of faces, rather than the schematic faces illustrated in the Fig.
Unbeknownst to participants, match feedback did not depend on which side of the coin they chose. Instead, three opponents (randomly determined) provided match feedback on 80% of trials. The remaining opponents provided match feedback on 60% of trials. Two opponents (one 80% and one 60%) always displayed genuine smiles on match trials, two displayed polite smiles (one 80% and one 60%) and the others remained neutral (Fig 1). These reward values remained the same across exposure and test phases. Participants experienced three blocks of 60 exposure-trials, viewing each opponent 10 times per block in random order.
The test phase allowed us to determine the degree to which participants valued genuine and polite smile feedback in monetary terms. When given a choice, between a pair of opponents, the degree to which a participant is willing to forgo the chance to win money in favour of the chance to see each smile indicates that smile’s subjective value to the participant. On each trial, participants chose the “most valuable” opponent from amongst a pair of neutrally posed opponents (Fig 1b). After participants chose an opponent, trials continued as in the exposure phase. All fifteen possible opponent pairings appeared eight times each, in random order (120 test-trials), with each face in a pair appearing as often on the left as on the right in the pairing.
Participants’ choices served as our measure of their preferences for social, relative to monetary value in this experiment. That is, if participants genuinely prefer one face to another in a given pairing, they will choose that one more often. If they are indifferent with respect to which face they prefer in the pairing, they will choose each face in that pairing with similar frequency. For example, if a participant strongly prefers smiling to neutral faces, that participant will select smiling faces whenever they are asked to choose between a face that smiles and a face that does not. This will be true even if the smiling face has a lower probability of giving a monetary reward. Based on the relative differences between the monetary and social values of the faces in each pairing, and participants’ preferences for one face versus another, it is possible to estimate how much a participant values smiles to neutral faces, expressed in monetary terms.
To ensure that specific opponent/value pairings did not affect results, each opponent’s face appeared in each monetary/social value combination with approximately equal frequency across participants. Half the participants saw male opponents and half saw females, counterbalanced by participant gender. The experiment was programmed using the Psychophysics Toolbox extensions  for MATLAB (The Mathworks).
Smile discrimination task
Participants additionally completed a smile discrimination task to ensure that any differences in choice behaviour did not relate to the ability to discriminate genuine from polite smiles. Participants viewed a series of 24 still images of faces (12 female) displaying either genuine or polite smiles and indicated with a key-press which type of smile the face displayed.
Because participants experienced all opponent pairings, it was possible to determine how different opponent features changed participants’ selections based on differences in the features of the selected and unselected opponents within a given pairing. We individually estimated the degree to which money (lower versus higher value) and smiles (genuine versus neutral; polite versus neutral) influenced participants’ decisions in the test phase of the smile-valuation task by applying Utility Theory  to the choice data (see also [15, 41]. This analysis allowed us to estimate the subjective value of the differences between opponents’ social and monetary values in a given opponent-pairing by quantifying how these differences shaped participants’ choice behaviour. Each participant’s raw choice data were individually fit using the logistic response function: (1) where POpponent A is the probability of choosing the left over the right opponent in an opponent pair (see Fig 1b), and θ is the difference in opponents’ utilities, modelled by the linear function: (2)
In this equation, XMoney codes the difference between the left and right opponents’ expected monetary values. A stimulus’s expected value is its win value multiplied by its win probability ; in this case, .02GBP multiplied by either an 80% or a 60% chance of winning. Thus, XMoney received a value of .4 if the left opponent was better than the right, -.4 if the right opponent was better, and 0 if they were equal. XGenuine coded genuine smiles. If the left opponent smiled genuinely and the right was neutral, we coded this as 1, if expression types were reversed, we coded this as -1, and if both or neither opponent smiled genuinely, this was coded 0. XPolite was coded in similar fashion for polite smiles. The βs are the unstandardized logistic regression weightings for each model component. These were estimated using a robust, iteratively re-weighted least squares algorithm to obtain the maximum likelihood estimates for each term in the model . The regression coefficients from this model were examined statistically using MANOVA with feedback type (acceptance, negative non-rejection, rejection) as the independent variable.
Our smile discrimination task data were analyzed using a signal detection framework. We recorded a hit when participants correctly identified a genuine smile and a false alarm for a polite smile identified as genuine. We computed d’ as a measure of participants’ ability to discriminate between the smiles. We estimated d’ with the following formula: (3) where ZH is the z-transformed probability of a hit and ZF is the z-transformed probability of a false alarm. We also calculated a measure of the degree to which participants’ responses were biased toward one smile or the other (this measure is known as “criterion” in signal detection theory terms, ) and was calculated as: (4) with ZH and ZF as noted above. This allowed us to determine whether our manipulation altered participants’ ability to identify smiles, as previous research suggests , or introduced bias into their responses on our discrimination task. All post-hoc tests were Bonferroni corrected for multiple comparisons.
To ensure that the participants who received rejection feedback were the only participants who experienced changes in social state (feelings of rejection), we examined self-reported post-manipulation social state, controlling for pre-manipulation social state in an ANCOVA model (see Fig 2). The omnibus test showed significant changes from baseline feelings of rejection, F(2,86) = 5.557, p = .005, η2p = .114. Post-hoc analyses showed that participants who received rejection feedback reported a significant increase in feelings of rejection relative to both acceptance (p = .029) and negative control conditions (p = .008). These data therefore suggest that only the rejection manipulation altered social state.
Affect change (positive, negative and feelings of rejection) from pre- to post-manipulation. Error bars show the 95%CIs.
Fig 3 shows the proportion of choices participants allocated to each face in a given pairing. A MANOVA model examining differences in the regression weights across conditions showed that there were no differences in the degree to which money influenced choice behaviour, regardless of the social feedback participants received, F(2,87) = 1.399, p = .252, η2p = .031 (Fig 4). Based on the fact that the regression coefficients were significantly greater than zero (t(29)-values>5.074, p-values<.001), all participants treated genuine smiles as valuable stimuli. There were no group differences in how they treated these smiles, F(2,87) = .957, p = .388, η2p = .022.
Average proportion choices allocated to stimuli in various pairings in the smile valuation task depending on value (holding expression constant) and expressive display (holding value constant) across feedback groups. Error bars show the 95%CIs.
Violin plots of regression weights estimated by the logistic model, indicating the degree to which money, genuine smiles, and polite smiles influenced choice behaviour across conditions. The grey-shaded central boxes show the inter-quartile range and the whiskers show the 95th percentile of the data distribution. Plot upper and lower boundaries show the full range of the data. The white dots depict the distribution medians while the notches show the 95% confidence interval on the medians. Individual data points are marked with coloured dots on each plot.
In contrast, the value of polite smiles was specifically influenced by the rejection feedback, F(2,87) = 14.563, p<.001, η2p = .251 (Fig 4). Participants receiving acceptance and negative, non-rejection feedback treated polite smiles as positive stimuli (regression coefficients significantly greater than zero; t(29)-values>2.831, p-values<.008). These groups did not differ from one another (p = .715). For participants receiving rejection feedback polite smiles became aversive stimuli (regression coefficient smaller than zero; t(29) = -3.023, p = .005). This value differed significantly from both other social feedback groups (p-values<.001). Thus, social rejection influenced the value of polite, but not genuine smiles and, although polite smiles influenced other participants’ choices positively, polite smiles lost their value for participants in the heightened state of social need participants experienced following rejection feedback.
Comparing the regression weights for genuine and polite smiles with those for money allowed us to estimate the subjective value of a smile in monetary terms. The value of a genuine smile was 1.390 pence (95%CI = [.830, 1.950]), .972 pence (95%CI = [.616, 1.328]), or 1.617 pence (95%CI = [1.020, 2.214]) for the acceptance, negative-non-rejection and rejection groups respectively. Polite smiles were worth 1.026 pence (95%CI = [.536, 1.517]) and 0.661 pence (95%CI = [.183, 1.138]) for the acceptance and negative non-rejection groups. For rejected participants however, polite smiles were worth -.849 pence (95%CI = [-1.423, -.275]), meaning that the financial value of politely smiling faces would need to be about .85 pence higher than that of a neutral face to achieve an equal probability of the politely smiling face being chosen (point of indifference).
Finally, we examined perceptual sensitivity for genuine and polite smiles in a short smile discrimination task in which participants differentiated these smiles in order to determine whether potential differences in smile discrimination ability affected results. A MANOVA with d’ and criterion as the dependent variables and feedback type as the independent variable showed that there were no group differences in participants’ ability to differentiate genuine from polite smiles, F(2,87) = .340, p = .713, η2p = .008 (Average d’: Acceptance = 1.856 [SD = 1.597]; Negative non-rejection = 1.608 [SD = .845]; Rejection = 1.592 [SD = 1.590]), nor did the groups differ in the degree to which they were biased to say a smile was genuine, F(2,87) = 0.623; p = .539, η2p = .014 (Average criterion: Acceptance = .009 [SD = .603]; Negative non-rejection = -.099 [SD = .455]; Rejection = -.141 [SD = .548]). Thus, the differences in participants’ choice behaviour were not due to alterations in smile-discrimination ability.
Contrary to prediction, the main finding from Experiment 1 is that participants expecting to interact with a rejecting partner did not increase the degree to which they valued genuine smiles. Rather, genuine smiles held their value whereas polite smiles changed from positive to negative cues for these participants. Thus, although the difference between genuine and polite smiles increased dramatically for participants expecting rejection, this was entirely due to decreases in the valuation of polite smiles. A control experiment (see Supporting Information: Figures A and B in S1 File) showed that a simple positive/negative affect manipulation was not sufficient to alter decision-making in response to smiles. Thus, these data suggest that polite-smile-value is malleable and that polite smile devaluation appears to be specific to the altered social state our manipulation provoked. This finding suggests an important prediction for how changes in social state might alter nonverbal behaviour in face-to-face social environments. Specifically, participants who believe they are interacting with a potential rejecter should be less motivated/likely to respond to that person’s polite smiles, a prediction we now test.
If rejected participants do indeed devalue polite smiles, they should reciprocate a lower proportion of these smiles during a social interaction with a potential rejecter, relative to participants who do not expect rejection. Additionally, based on the fact that we did not see differences in genuine smile valuation in Experiment 1, we predict that genuine smile reciprocity should be unaffected by the manipulation.
We recruited 144 participants (mean age: 21.90, SD = 2.64) for a study of “personality characteristics and first impressions.” Participants were pseudo-randomly assigned to same-sex dyads with the constraint that the age gap between participants be 3 years or less. There were 36 male and 36 female dyads. The number of dyads recruited was determined a priori, based on the Experiment 1 effect size (effect size f = .587, α = .05, 1-β = .95). The sample-size calculation suggested 16 dyads per condition. However, we again oversampled to achieve 24 dyads per condition after exclusions. This was the maximum we were able to recruit given time and financial constraints (i.e., the ability to get the data coded without external funding for the project). Participants received 7.00GBP for taking part in the study. We excluded data from seven additional dyads in which the target participant did not believe the feedback (four dyads); the dyad members knew one another (one dyad); or technical difficulties interfered with video recording (two dyads). As in Experiment 1, all participants scored less than 60 on the Interaction Anxiousness Scale.
Upon arrival, an experimenter escorted participants to separate rooms in which they completed the same personality questionnaires and manipulation as in Experiment 1. In this case however, only one ‘target’ member of each dyad (randomly assigned) received active feedback. The other member received no feedback. This meant that these ‘neutral’ participants did not approach the interaction with any manipulated expectations about interaction outcomes. The inclusion of neutral participants allowed us to determine how a social partner might perceive behaviour changes on the part of target participants.
Experiment 2 used the same three feedback conditions as Experiment 1. The feedback types were pseudo-randomly assigned to dyads such that 24 dyads received each feedback condition and half the dyads per condition were male pairs. Because the computer controlled feedback assignment, the experimenter remained blind to both which participants received active/neutral feedback and which type of active feedback each dyad received.
Following the manipulation the experimenter escorted participants to an interaction room, introduced them, and left them alone with the instruction to converse for a few minutes on any topic. Conversations lasted 5 minutes and were captured on digital video. After the interaction, the experimenter returned participants to their individual rooms to complete a modified Desire for Future Interaction scale (DFI ) as a measure of partner liking. The DFI assesses the degree to which participants desire future meetings with the partner (e.g., “I would like to meet my partner again.”) using a 6-point Likert scale (1 = definitely no; 6 = definitely yes; present sample internal consistency: α = .84). Thereafter, the experimenter probed participants for suspicion, debriefed and paid them, explained the manipulation and allowed them to provide fully informed consent or have their data destroyed. No participant declined consent.
Four research assistants independently coded participants’ smiles. All were naive to study hypotheses and participants’ feedback conditions. Before coding any data, coders were trained to 95% agreement on a set of pilot interactions. To check post-training reliability, each coder independently coded a series of three interactions, achieving 94% agreement. For the remaining interactions, two coders overlapped their coding for 50% of the sessions (36 sessions, both participants).
Coders viewed one participant at a time (video only; a computer program suppressed the audio files during coding) and coded that participant’s behaviour for the entire 5-minute interaction before coding another participant’s data. We recorded interactions at a rate of 25 frames-per-second and used frame counts to identify the onsets of each smile and to link dyad members’ behaviour. Each time they saw a smile, coders noted the frame at onset, frame at offset and type of smile produced. To differentiate the smiles, coders used a FACES-based coding system  and classified smiles as genuine if they involved the eye region and polite if they did not. Cohen’s kappa coefficients, which assess inter-rater agreement, showed excellent reliability for both smile types (genuine: .93, polite, .91, see ). Coder disagreements were resolved by consensus.
To determine instances of smile reciprocity for each target participant, we used a purpose-written MATLAB script to examine the coded sessions for each smile that the neutral partner initiated (i.e., the target participant was not smiling and the neutral partner began smiling) and marked a reciprocated smile if 1) the target participant smiled back at the partner within four seconds (see ), and 2) the return smile was the same smile type as the initiating smile (genuine or polite). We used these data to calculate the proportion of partner smiles each participant returned for each smile type.
The design included four participant types (neutral, acceptance, rejection, negative non-rejection), nested within three dyad types (acceptance, rejection, negative non-rejection). Because we were focused specifically on the effects our social state manipulation had on the proportion of genuine and polite smiles participants returned, we excluded the neutral partners from the analysis of smile reciprocity. This effectively removed the source of the interdependence between participants nested within a given dyad (see [58, 59–61]). All post-hoc comparisons used Bonferroni’s correction for Type I error.
As Fig 5 shows, target participants reciprocated similar proportions of their partners’ genuine smiles across feedback conditions, F(2,69) = 1.286, p = .283, η2p = .036, meaning that as anticipated, the feedback did not affect genuine smile reciprocity across participant-types. However, when we examined the proportion of polite smiles target participants returned, the predicted effect was significant, F(2,69) = 10.136, p<.001, η2p = .227. These results suggest that participants who interacted with a partner they believed had rejected them returned a significantly lower proportion of their partners’ polite smiles relative to those who believed that the partner liked them (p<.001) and those who received negative non-rejection feedback (p = .002). These data suggest that our social-state manipulation shaped real-world social behaviour in a manner consistent with the Experiment 1 findings. That is, participants reduced their polite smile reciprocity in an interaction with a potential rejecter, without altering genuine smile reciprocity. These results therefore serve as a conceptual replication of Experiment 1.
Violin plots showing percent genuine and polite smiles returned in face-to-face interaction depending on participant condition. The grey-shaded central boxes show the inter-quartile range and the whiskers show the 95th percentile of the data distribution. Plot upper and lower boundaries show the full range of the data. The white dots depict the distribution medians while the notches show the 95% confidence interval on the medians. Individual data points are marked with coloured dots on each plot.
One important question that this study methodology allows us to ask relates to how much neutral participants notice such seemingly subtle changes in behaviour. To investigate this question, we examined how much participants reported liking their partners. Unsurprisingly, target participants in acceptance dyads liked their partners better than did participants in other dyad types (Fig 6a; p-values<.021; Omnibus F(2,69) = 5.604, p = .006, η2p = .140), who did not differ from one another (p>.999). Interestingly, neutral partners’ ratings of targets showed a similar pattern (Omnibus F(2,69) = 3.770, p = .028, η2p = .099; acceptance > rejection: p = .034; no other comparisons were significant after Bonferroni correction: p-values>.132). To understand whether participants’ smile reciprocity predicted this effect, we selected only target participants and regressed their feedback-type, self-reported liking for the partner, and genuine and polite smile reciprocity on the partner’s liking for them in a hierarchical model (Table 1; Fig 6b). Results showed that after accounting for feedback-type and participants’ own liking ratings, polite smile reciprocity predicted the degree to which the neutral partner liked them. The effect of genuine smile reciprocity was not significant. These results suggest that as in previous research  this seemingly small change in social behaviour was both noticeable to neutral partners and has important social consequences related to the likelihood of future relationship development.
a) Self-reported liking of the partner across participant conditions, including both neutral (hatched bars) and target participants within dyads. Error bars show the 95%CIs. b) Relationship between proportion of genuine and polite smiles participants returned and the partner’s liking rating.
Together, these results suggest that a social cue’s subjective value affects the moment-to-moment social decisions people make in response to that cue. Specifically, participants induced to feel states of high social need based on receiving information about possible rejection demonstrated significant polite smile devaluation and reduced reciprocity for polite smiles during naturalistic face-to-face interaction. The fact that complementary findings occurred in a computer-based economic task and in participants’ moment-to-moment social decisions is, to our knowledge, the first direct evidence that social-state dependent modulation of social-cue utility has real-world consequences. This finding suggests that reward-based decision-making may govern at least some of the moment-to-moment behaviours people produce in natural social environments.
How might such decision-making work? One suggestion from the literature is that people pay more attention to cues associated with high-value outcomes [63, 64]. Insofar as this is true, it might be the case that people are less likely to attend to low-value behaviours and consequently less likely to respond to those cues [10, 65]. However, because people generally experience predictable smile reciprocity from their partners [10, 37], reduced smile-return rates may be experienced as unpleasant and eventually cause discomfort and dislike.
One major implication of this set of findings is that neuroeconomic approaches to social stimuli may offer real opportunities for predicting how internal experiences such as emotions drive both the social behaviour people produce and their evaluations of partners’ behaviour. Specifically, neuroeconomic methods allow the quantification of relative stimulus utility more precisely than traditional psychological methods involving Likert rating scales. These methods therefore render the highly subjective social stimuli people encounter, such as smiles, tractable to prediction under formal mathematical models. This is a necessary step in understanding how the neural systems that underpin social decision-making function to control face-to-face social behaviour.
Importantly, the general affective changes associated with anticipating social rejection cannot explain our findings. Participants in the negative non-rejection condition, who did not experience alterations in social state, showed no changes in social decision-making although they reported similar affect changes as participants anticipating rejection. This implies that a specific social state manipulation is required to alter the utility of a polite smile. Data from our control study (Supporting Information: Figures A and B in S1 File) confirm this result, showing that a simple emotion induction is not sufficient to alter smile utility.
Our results cannot be due to social-state-dependent alterations in the ability to differentiate smiles because our manipulation did not alter participants’ ability to distinguish genuine from polite smiles, nor did it induce response bias. This finding runs contrary to previous findings (e.g., ). However, we note that our smile stimuli utilize a much more controlled set of still images, selected based on the degree to which they were discriminable, rather than the video smile-stimuli used in previous research.
As in much social psychological research, we have estimated social value in the relatively abstract setting of the laboratory, by measuring decisions to static face stimuli on a computer screen. However, unlike the majority of research in this field, we also show that these laboratory findings specifically predict behaviour in naturalistic social interactions. We think this combination of methodologies, in which we examine a prediction from the laboratory in the more complex world of face-to-face social interaction, is a critical step toward ensuring the ecological validity of psychological results.
Of course, the face-to-face interactions we report are limited by the fact that they took place only between target participants and people they believed had directly evaluated and possibly rejected them. Thus, our findings may not converge neatly with other rejection-related findings showing mimicry increases after rejection (e.g., ) because one would expect to see less prosocial behaviour during interactions with rejecters [21, 49] compared to neutral or inclusive individuals as in . We also examined only one type of social state manipulation–social rejection. It is therefore important that future research examine the influence of different social-state manipulations on both cue valuation and social behaviour. Nonetheless, our findings suggest that cue-value models provide leverage for understanding how typical participants make behavioural decisions in face-to-face interactions.
In summary, we demonstrate evidence of social-state-dependent fluctuations in the degree to which people subjectively value social rewards. Our findings show that both genuine and polite smiles are rewarding, relative to neutral faces. However, in states of high social need, polite smiles turn from mildly positive into negative social stimuli. This effect translates to real-world social decisions by reducing the frequency with which those anticipating social rejection respond to a social partner’s polite smiles. Thus, the degree to which a social cue influences social decisions depends profoundly on the receiver’s social state and suggests that understanding how people subjectively value the social stimuli they receive may provide important evidence for understanding how the brain controls and tunes the moment-to-moment production of social behaviour.
S1 Fig. False feedback pilot data.
Pilot participants’ perceptions of acceptance, rejection and negative non-rejection feedback. Error bars show the 95%CI. P-values for feedback-type comparisons are Bonferroni corrected.
We wish to thank our coders for their help with smile coding. De-identified raw and summary data (Study 1) and coded interaction data (Study 2) may be obtained from https://osf.io/mxt63/ (interaction videos are not available, as participants have not consented to the release of image/audio files).
- 1. Schilbach L, Timmermans B, Reddy V, Costall A, Bente G, Schlicht T, et al. Toward a second-person neuroscience. The Behavioral and brain sciences. 2013;36(4):393–414. Epub 2013/07/26. pmid:23883742.
- 2. Frank MG, Ekman P, Friesen WV. Behavioral markers and recognizability of the smile of enjoyment. Journal of personality and social psychology. 1993;64(1):83–93. Epub 1993/01/01. pmid:8421253.
- 3. Maringer M, Krumhuber EG, Fischer AH, Niedenthal PM. Beyond smile dynamics: mimicry and beliefs in judgments of smiles. Emotion. 2011;11(1):181–7. Epub 2011/03/16. pmid:21401238.
- 4. Frank MG, Ekman P. Not all smiles are created equal: The differences between enjoyment and nonenjoyment smiles. International Journal of Humor Research. 1993;6(1):9–26.
- 5. Kringelbach ML, Rolls ET. Neural correlates of rapid reversal learning in a simple model of human social interaction. NeuroImage. 2003;20(2):1371–83. Epub 2003/10/22. pmid:14568506.
- 6. O’Doherty J, Winston J, Critchley H, Perrett D, Burt DM, Dolan RJ. Beauty in a smile: the role of medial orbitofrontal cortex in facial attractiveness. Neuropsychologia. 2003;41(2):147–55. Epub 2002/12/03. pmid:12459213.
- 7. Ekman P, Davidson RJ, Friesen WV. The Duchenne smile: emotional expression and brain physiology. II. Journal of personality and social psychology. 1990;58(2):342–53. Epub 1990/02/01. pmid:2319446.
- 8. Calvo MG, Marrero H, Beltran D. When does the brain distinguish between genuine and ambiguous smiles? An ERP study. Brain and cognition. 2013;81(2):237–46. Epub 2012/12/25. pmid:23262178.
- 9. Goffman E. On face-work; an analysis of ritual elements in social interaction. Psychiatry. 1955;18(3):213–31. Epub 1955/08/01. pmid:13254953.
- 10. Heerey EA, Crossley HM. Predictive and reactive mechanisms in smile reciprocity. Psychological science. 2013;24(8):1446–55. Epub 2013/06/08. pmid:23744875.
- 11. Hess U, Bourgeois P. You smile—I smile: emotion expression in social interaction. Biological psychology. 2010;84(3):514–20. Epub 2009/11/17. pmid:19913071.
- 12. Montoya RM, Kershaw C, Prosser JL. A meta-analytic investigation of the relation between interpersonal attraction and enacted behavior. Psychological bulletin. 2018;144(7):673–709. pmid:29733622
- 13. Krumhuber E, Manstead AS, Cosker D, Marshall D, Rosin PL, Kappas A. Facial dynamics as indicators of trustworthiness and cooperative behavior. Emotion. 2007;7(4):730–5. Epub 2007/11/28. pmid:18039040.
- 14. Scharlemann JPW, Eckel CC, Kacelnik A, Wilson RK. The value of a smile: Game theory with a human face. J Economic Psychol. 2001;22(5):617–40.
- 15. Shore DM, Heerey EA. The value of genuine and polite smiles. Emotion. 2011;11(1):169–74. Epub 2011/03/16. pmid:21401236.
- 16. Glimcher PW, Dorris MC, Bayer HM. Physiological utility theory and the neuroeconomics of choice. Games and economic behavior. 2005;52(2):213–56. Epub 2006/07/18. pmid:16845435
- 17. Levy DJ, Thavikulwat AC, Glimcher PW. State dependent valuation: the effect of deprivation on risk preferences. PloS one. 2013;8(1):e53978. Epub 2013/01/30. pmid:23358126
- 18. Baumeister RF, Leary MR. The need to belong: desire for interpersonal attachments as a fundamental human motivation. Psychological bulletin. 1995;117(3):497–529. Epub 1995/05/01. pmid:7777651.
- 19. Eisenberger NI, Lieberman MD, Williams KD. Does rejection hurt? An FMRI study of social exclusion. Science. 2003;302(5643):290–2. Epub 2003/10/11. pmid:14551436.
- 20. Kross E, Berman MG, Mischel W, Smith EE, Wager TD. Social rejection shares somatosensory representations with physical pain. Proceedings of the National Academy of Sciences of the United States of America. 2011;108(15):6270–5. pmid:21444827
- 21. Williams KD. Ostracism. Annual review of psychology. 2007;58:425–52. Epub 2006/09/14. pmid:16968209.
- 22. Eisenberger NI. The pain of social disconnection: examining the shared neural underpinnings of physical and social pain. Nature reviews Neuroscience. 2012;13(6):421–34. Epub 2012/05/04. pmid:22551663.
- 23. Macdonald G, Leary MR. Why does social exclusion hurt? The relationship between social and physical pain. Psychological bulletin. 2005;131(2):202–23. Epub 2005/03/03. pmid:15740417.
- 24. Twenge JM, Catanese KR, Baumeister RF. Social exclusion causes self-defeating behavior. Journal of personality and social psychology. 2002;83(3):606–15. Epub 2002/09/11. pmid:12219857.
- 25. Cacioppo S, Cacioppo JT. Research in social neuroscience: How perceived social isolation, ostracism, and romantic rejection affect our brain. In: Riva P, Eck J, editors. Social Exclusion. Switzerland: Springer, Cham; 2016. p. 73–88.
- 26. Achterberg M, van Duijvenvoorde AC, Bakermans-Kranenburg MJ, Crone EA. Control your anger! The neural basis of aggression regulation in response to negative social feedback. Social, Cognitive and Affective Neuroscience. 2016;11(5):712–20.
- 27. Leary MR. Responses to social exclusion: Social anxiety, jealousy, loneliness, depression and low self-esteem. Journal of Social and Clinical Psychology. 1990;9(2):221–9.
- 28. Jin ES, Josephs RA. Acute and Chronic Physiological Consequences of Social Rejection. In: Williams K, Nida SA, editors. Ostracism, Exclusion and Rejection. New York, NY: Routledge; 2016. p. 94–104.
- 29. Bernstein MJ, Sacco DF, Brown C, Young SG, Claypool HM. A preference for genuine smiles following social exclusion. Journal of experimental social psychology. 2010;46(1):196–9.
- 30. Dewall CN, Maner JK, Rouby DA. Social exclusion and early-stage interpersonal perception: selective attention to signs of acceptance. Journal of personality and social psychology. 2009;96(4):729–41. Epub 2009/03/25. pmid:19309198.
- 31. Kawamoto T, Nittono H, Ura M. Social exclusion induces early-stage perceptual and behavioral changes in response to social cues. Social neuroscience. 2014;9(2):174–85. pmid:24499456.
- 32. Pickett CL, Gardner WL, Knowles M. Getting a cue: the need to belong and enhanced sensitivity to social cues. Personality & social psychology bulletin. 2004;30(9):1095–107. Epub 2004/09/11. pmid:15359014.
- 33. Heerdink MW, Van Kleef GA, Homan AC, Fischer AH. Emotional expressions as social signals of rejection and acceptance: evidence from the affect misattribution paradigm. Journal of experimental social psychology. 2015;56(1):60–8.
- 34. Bernstein MJ, Young SG, Brown CM, Sacco DF, Claypool HM. Adaptive responses to social exclusion: social rejection improves detection of real and fake smiles. Psychological science. 2008;19(10):981–3. Epub 2008/11/13. pmid:19000206.
- 35. Maner JK, DeWall CN, Baumeister RF, Schaller M. Does social exclusion motivate interpersonal reconnection? Resolving the "porcupine problem". Journal of personality and social psychology. 2007;92(1):42–55. pmid:17201541.
- 36. Gardner WL, Pickett CL, Jefferis V, Knowles M. On the outside looking in: loneliness and social monitoring. Personality & social psychology bulletin. 2005;31(11):1549–60. pmid:16207773.
- 37. Heerey EA, Kring AM. Interpersonal consequences of social anxiety. Journal of abnormal psychology. 2007;116(1):125–34. Epub 2007/02/28. pmid:17324023.
- 38. Deaner RO, Khera AV, Platt ML. Monkeys pay per view: adaptive valuation of social images by rhesus macaques. Current biology: CB. 2005;15(6):543–8. pmid:15797023.
- 39. Levy DJ, Glimcher PW. Comparing apples and oranges: using reward-specific and reward-general subjective value representation in the brain. The Journal of neuroscience: the official journal of the Society for Neuroscience. 2011;31(41):14693–707. pmid:21994386
- 40. Levy DJ, Glimcher PW. The root of all value: a neural common currency for choice. Current opinion in neurobiology. 2012;22(6):1027–38. pmid:22766486
- 41. Catalano L, Heerey E, Gold JM. The valuation of social rewards in schizophrenia. Journal of abnormal psychology. 2018;In Press.
- 42. Horsky D, Rao MR. Estimation of attribute weights from preference comparisons. Management Science. 1984;30(7):801–22.
- 43. Marquardt R, Makens J, Larzelere H. Measuring the utility added by branding and grading. Journal of Marketing Research. 1965;2(1):45–50.
- 44. Faul F, Erdfelder E, Lang A, Buchner A. G*Power 3: A flexible statisticsl power analysis program for the social behavioral and biomedical sciences. Behavior research methods. 2007;39(2):175–91. pmid:17695343
- 45. Leary MR, Kowalski RM. The Interaction Anxiousness Scale: construct and criterion-related validity. Journal of personality assessment. 1993;61(1):136–46. Epub 1993/08/01. pmid:8377098.
- 46. Taylor CT, Bomyea J, Amir N. Attentional bias away from positive social information mediates the link between social anxiety and anxiety vulnerability to a social stressor. Journal of anxiety disorders. 2010;24(4):403–8. Epub 2010/03/09. pmid:20207102.
- 47. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. Journal of personality and social psychology. 1988;54(6):1063–70. Epub 1988/06/01. pmid:3397865.
- 48. Gunther Moor B, Crone EA, van der Molen MW. The heartbrake of social rejection: heart rate deceleration in response to unexpected peer rejection. Psychological science. 2010;21(9):1326–33. pmid:20696852.
- 49. Twenge JM, Baumeister RF, DeWall CN, Ciarocco NJ, Bartels JM. Social exclusion decreases prosocial behavior. Journal of personality and social psychology. 2007;92(1):56–66. Epub 2007/01/05. pmid:17201542.
- 50. Brainard DH. The Psychophysics Toolbox. Spatial vision. 1997;10(4):433–6. Epub 1997/01/01. pmid:9176952
- 51. von Neumann J, Morgenstern O. Theory of Games and Economic Behavior. Princeton, NJ: Princeton Univeristy Press; 1953.
- 52. Sutton RS, Barto AG. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press; 1998.
- 53. Daubechies I, DeVore R, Fornasier M, Guentuerk CS. Iteratively reweighted least squares minimization for sparse recovery. Communications on Pure and Applied Mathematics. 2009;63(1):1–38.
- 54. Macmillan NA, Creelman CD. Detection Theory: A User’s Guide. 2nd ed. Mahwah, New Jersey: Lawrence Erlbaum Associates; 2005.
- 55. Coyne JC. Depression and the response of others. Journal of abnormal psychology. 1976;2:186–93.
- 56. Kring AM, Sloan DM. The Facial Expression Coding System (FACES): development, validation, and utility. Psychological assessment. 2007;19(2):210–24. Epub 2007/06/15. pmid:17563202.
- 57. Bakeman R, Quera V. Sequential analysis and observational methods for the behavioral sciences. Cambridge, UK: Canbridge University Press; 2011.
- 58. Kenny DA, La Voie L. Separating individual and group effects. Journal of personality and social psychology. 1985;48:339–48.
- 59. Kenny DA, Judd CM. Consequences of violating the interdependence assumption in analysis of variance. Psychological bulletin. 1986;99:422–31.
- 60. Kenny DA, Kashy DA, Cook WL. Dyadic Data Analysis. New York, NY: Guilford; 2006.
- 61. Kenny DA, Mohr CD, Levesque MJ. A social relations variance partitioning of dyadic behavior. Psychological bulletin. 2001;127(1):128–41. Epub 2001/03/29. pmid:11271751.
- 62. Snyder M, Tanke ED, Bercheid E. Social perception and interpersonal behavior: On the self-fulfilling nature of social stereotypes. Journal of personality and social psychology. 1977;35(9):656–66.
- 63. Anderson BA. Social reward shapes attentional biases. Cogn Neurosci. 2016;7(1–4):30–6. pmid:25941868
- 64. Yantis S, Anderson BA, Wampler EK, Laurent PA. Reward and attentional control in visual search. Nebraska Symposium on Motivation Nebraska Symposium on Motivation. 2012;59:91–116. pmid:23437631
- 65. Shore DM, Heerey EA. Do social utility judgments influence attentional processing? Cognition. 2013;129(1):114–22. Epub 2013/07/28. pmid:23887150.
- 66. Cheung EO, Slotter EB, Gardner WL. Are you feeling what I’m feeling? The role of facial mimicry in facilitating reconnection following social exclusion. Motivation & Emotion. 2015;39(4):613–30.