http://orcid.org/0000-0002-5985-5658
Figures
Abstract
The importance of social connection to well-being is underscored by individuals’ reactivity to events highlighting the potential for rejection and exclusion, which extends even to observing the social exclusion of others (“vicarious ostracism”). Because responses to vicarious ostracism depend at least in part on empathy with the target, and individuals tend to empathize less readily with outgroup than ingroup members, the question arises as to whether there is a boundary condition on vicarious ostracism effects whereby individuals are relatively immune to observing ingroup-on-outgroup ostracism. Of particular interest is the case where members of a dominant ethnic group observe fellow ingroup members ostracize a member of a disadvantaged ethnic minority group, as here there is a compelling potential alternative: Perceived violation of contemporary social norms condemning prejudice and discrimination might instead lead dominant group members to be especially upset by “dominant-on-disadvantaged” ostracism. Accordingly, the present research examines, across four studies and 4413 participants, individuals’ affective reactions to observing dominant-on-disadvantaged versus dominant-on-dominant ostracism. In each study, dominant group members (White/Europeans) observed dominant group members include or ostracize a fellow dominant group member or a disadvantaged ethnic minority group member (a Black individual) in an online Cyberball game. Results revealed that dominant group members felt more guilt, anger, and sadness after observing severe ostracism of a disadvantaged as opposed to dominant group member. Although no direct effects emerged on behavioral outcomes, exploratory analyses suggested that observing ostracism of a disadvantaged (versus dominant) group member had indirect effects on behavior via increased feelings of anger. These results suggest that observing ostracism may be a sufficiently potent and relatable experience that when it occurs across group boundaries it awakens individuals’ sensitivity to injustice and discrimination.
Citation: Petsnik C, Vorauer JD (2020) Do dominant group members have different emotional responses to observing dominant-on-dominant versus dominant-on-disadvantaged ostracism? Some evidence for heightened reactivity to potentially discriminatory ingroup behavior. PLoS ONE 15(6): e0234540. https://doi.org/10.1371/journal.pone.0234540
Editor: Jan De Houwer, Ghent University, BELGIUM
Received: November 20, 2019; Accepted: May 26, 2020; Published: June 25, 2020
Copyright: © 2020 Petsnik, Vorauer. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are available in OSF: (osf.io/wg7h9).
Funding: This research was supported by a Canada Graduate Scholarship from the Social Sciences and Humanities Research Council of Canada (https://www.sshrc-crsh.gc.ca) and a Tri Council Master’s Supplement Award from the University of Manitoba (http://umanitoba.ca) awarded to Corey Petsnik as well as a Social Sciences and Humanities Research Council of Canada grant (grant number 435-2012-0348) awarded to Jacquie D. Vorauer. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Social connections are profoundly important to mental and physical well-being [1]. Moreover, relations and interactions with others help ground a fundamental aspect of individuals’ self-perception: Individuals’ own feelings of self-esteem are heavily intertwined with their feelings of being socially accepted or rejected [2]. Consequently, individuals are highly sensitive to information that even only remotely reflects on their social standing with others. Attesting to the deeply ingrained and automatic nature of such sensitivity, it applies even in cases where negative treatment is plausibly directed toward others instead of oneself, or comes from an unimportant or despised other or a computer [3–7].
This sensitivity goes further yet: Individuals who observe another person being ostracized–even when there is no potential for personally being a target of mistreatment themselves–experience reduced satisfaction of fundamental needs (e.g., belonging and self-esteem) and more negative mood [8]. Such “vicarious ostracism” has been found across observers of various ages, ranging from children to adults [9] and at times appears to be just as potent as being directly targeted [10].
Beyond these intrapersonal effects, vicarious ostracism has also been linked to a variety of interpersonal outcomes. In particular, observing ostracism may often motivate prosocial behavior by observers toward the targeted individual, perhaps with the intention of alleviating their distress. For instance, observers actively seek to reinclude ostracized individuals, write excluded others encouraging emails, and in economic games sacrifice some of their own resources to compensate the target [8,9,11,12]. Moreover, in addition to assisting ostracized individuals, studies have found that observers will also punish the perpetrators (e.g., by allocating them less money [8,9,13,14]).
However, it is not always the case that individuals are distressed by observing ostracism and subsequently act to assist the target and discipline the perpetrators. Indeed, research indicates that under certain conditions observers are not only unperturbed by a target’s ostracism, reporting, for instance, feeling little anger about the target’s mistreatment or sympathy toward him or her, but are also motivated to blame and punish the target, and may even join the perpetrators in ostracizing them [15–19].
The present research aims to further elucidate some of the factors that might make observers more or less sensitive to witnessing the ostracism of another individual. Specifically, we examine the effect of a target’s ethnic group membership on observers’ reactions to ostracism, with a particular focus on how dominant ethnic group members react to observing other dominant group members perpetrate ostracism against a disadvantaged ethnic minority group member (“dominant-on-disadvantaged” ostracism) relative to a fellow dominant group member (“dominant-on-dominant” ostracism). Combining theory and research on the effect of group membership on observers’ reactions to ostracism with that on intergroup perception, affect, and relations we propose three alternative hypotheses for how observers might react to dominant-on-disadvantaged versus dominant-on-dominant ostracism: Reduced reactivity, enhanced reactivity, or equal reactivity. We then test these competing hypotheses in a “mini-meta-analysis” [20] across four online experiments.
Reduced reactivity hypothesis
Research indicates that vicarious ostracism is enhanced when individuals actively try to take the perspective of the target [21] or are high in dispositional empathy [22]. These findings point to a readiness to identify and empathize with targets of ostracism as a key underlying mechanism. The important role played by empathy in driving individuals’ reactivity to others’ mistreatment also suggests a potential boundary condition: Perhaps when the witnessed ostracism involves ingroup members ostracizing an outgroup member individuals’ sensitivity is shut down. Such reduced reactivity would make sense from a variety of perspectives. Most notably, ample research suggests an “empathy gap” across group boundaries whereby individuals are less likely to empathize with outgroup than ingroup members in a range of different circumstances, such as when observing another person’s expressions of sadness [23], witnessing another individual’s experience of physical pain [24,25], or learning of negative events or misfortunes another person has experienced [26]. Deficits in empathy have been identified across various naturally occurring intergroup boundaries (e.g., political or ethnic differences) as well as boundaries created in the laboratory, indicating that an empathy gap for outgroup members is a reliable and robust phenomenon [23,27–29]. Together with research indicating an important role for empathy in vicarious ostracism, this evidence for an empathy deficit with respect to outgroup members, including those that may belong to a different and potentially disadvantaged ethnic group [30], suggests that dominant group members should have less negative reactions to seeing dominant group members ostracize a disadvantaged group member as compared to a fellow dominant group member.
Other forces may also contribute to dominant group members' reduced reactivity to observing dominant-on-disadvantaged ostracism. In particular, the motivation to maintain a positive social identity should presumably push these individuals toward downplaying the negativity of the target’s treatment, perhaps by considering it as provoked or justified [31]. Relatedly, dampened reactions may be particularly evident when the observer and perpetrators share an advantaged group membership, the target is a member of a disadvantaged group, and the former perceive their advantage as legitimate [32,33]. This may be especially likely if advantaged observers feel that as result of their association with the perpetrators they are being unfairly accused of being prejudiced [34].
The hypothesis that dominant group members will react less negatively when witnessing ingroup members ostracize a disadvantaged group member as compared to a fellow dominant group member, which we refer to as the reduced reactivity hypothesis, is also in line with research suggesting bystander apathy, in both conscious self-reports and physiological responding, with respect to witnessing explicitly racist and homophobic comments directed toward outgroup targets [35–37].
Although only limited research has directly evaluated the impact of group membership on observers’ reactions to ostracism, the results of some of these studies provide support for the hypothesis that dominant group members will be less distressed by witnessing dominant-on-disadvantaged relative to dominant-on-dominant ostracism. Notably, Veldhuis et al. [38] demonstrated that witnessing the ostracism of those who share one’s political preferences elicits greater feelings of humiliation relative to observing the ostracism of those with differing political leanings. Similarly, manipulating ingroup-outgroup status using a minimal group paradigm, Forbes et al. [39] found that observers were more likely to reinclude an ostracized ingroup member relative to an outgroup member. Additional research documenting that a peer relationship or close friendship with targets enhances observers’ empathic responses toward ostracized targets [40,41] and that a peer relationship or close friendship with perpetrators diminishes observers’ punishment of perpetrators [13,42], as well as work indicating that targets experience greater distress when they are ostracized by racial ingroup relative to outgroup members [43,44], is also broadly supportive of reduced reactivity to dominant-on-disadvantaged ostracism.
Enhanced reactivity hypothesis
However, a contrasting possibility, enhanced reactivity (enhanced reactivity hypothesis), is also supported by several lines of research and theorizing. Perceived violation of contemporary social norms condemning prejudice and discrimination [45] may lead individuals to be more upset by ostracism across as compared to within group boundaries. Enhanced perceptions of the unfairness and pervasiveness [46] of group-based treatment could conceivably contribute to such effects, perhaps especially in the case of ascribed characteristics such as sex and race. Negative reactions could be even more likely on the part of dominant group members when the ostracism is perpetrated by ingroup members against a member of an ethnic minority group because of the social identity threat represented by the possibility that their ingroup–and by extension they themselves–are racist or appear to be racist [47–49]. Consistent with this idea, witnessing but not confronting discriminatory behavior has been shown to be a potentially dissonance-arousing experience [50]. Further, prior research suggests that the salience of a dominant group identity may elicit a variety of negative group-based emotions if individuals appraise their ingroup as responsible for perpetrating harm or illegitimate advantage over an outgroup [51,52].
Some research on the impact of group membership on observers’ responses to ostracism also suggests heightened reactivity in the intergroup case. In particular, Rudert et al. [16] proposed the “social dissimilarity rule” whereby observers’ reactions to witnessing ostracism are governed by perceived similarity or dissimilarity between targets and perpetrators, such that when individuals observe exchanges in which a dissimilar person is ostracized they are inclined to devalue the ostracizers, sympathize with the target, and attribute the ostracism to malicious motives (e.g., prejudice or discrimination), whereas if a similar person is ostracized they devalue the target and attribute the ostracism to punitive motives. Rudert et al. found support for this hypothesis across a series of studies where similarity/dissimilarity was manipulated through shared group membership, including shared ethnic background. However, because in these studies participants did not share group memberships with ostracizers or targets–which could clearly affect their identifications and motivations–the implications for situations, such as dominant-on-disadvantaged ostracism, where ingroup members are observed ostracizing an outgroup member are unclear.
In another study examining observers’ judgments of ostracism more generally, Rudert et al. [53] found that the ostracism of individuals perceived as warm-and-incompetent was judged to be the least acceptable. As members of many disadvantaged groups tend to be viewed as more warm than competent [54], observers’ general beliefs about the attributes of a disadvantaged target may lead them to especially condemn their mistreatment. Further, research on targets’ reactions has revealed more reactivity to outgroup versus ingroup ostracism, at least on some measures. For instance, Schaafsma et al. [55] found that exclusion by outgroup members led to more hostility than exclusion by ingroup members, an effect that was mediated by attributions to racism, and Williams et al. [56] found that being ostracized by outgroup members reduced felt belonging more than did being ostracized by ingroup members.
Equal reactivity hypothesis
Of course another possibility is that the target’s group status just does not matter to individuals’ reactions. This could suggest that ostracism is such a fundamental and universal experience that observing it take place is sufficiently powerful and basic to overwhelm any effects associated with whether the target is an ingroup or an outgroup member. Perhaps when individuals can readily feel that “it could happen to me” they are poised to identify with and react to another’s experience on its own terms irrespective of the person’s group membership. Broadly consistent with this possibility is research by Arpin et al. [57] that found, using a minimal group paradigm, no impact of group membership on observers’ reactions to ostracism. Also relevant here is work on targets’ responses indicating similar levels of distress regardless of whether the perpetrators are ingroup or outgroup members [4,58,59].
Overview
We probed dominant group members’ reactions to dominant-on-dominant vs. dominant-on-disadvantaged ostracism in a series of four online experiments involving a total of 4413 participants. In each of the experiments White participants observed a computer-mediated ball-tossing game (“Cyberball” [56]) in which two White individuals either ostracized or included a third player, who was either a White or Black individual. Their affective and behavioral reactions and impressions of the players were then assessed.
Several of our design decisions warrant discussion. First, because our theoretical focus was on the implications of the target’s group membership for observers’ reactions to ostracism, it was necessary to hold the perpetrators’ group membership constant. Thus we considered dominant-on-dominant ostracism to be the most appropriate control condition against which dominant-on-disadvantaged ostracism could be compared. This comparison afforded by our studies is relatively unique. To our knowledge only one other study has compared ingroup-on-ingroup to ingroup-on-outgroup ostracism, that of Forbes et al. [39], and they did so using a minimal group paradigm. Indeed most previous work–which was focused on answering different questions than those of interest here–has either not considered the effects of a shared group membership between observers, ostracizers, and targets [16] or has probed the effects of observing outgroup-on-ingroup versus outgroup-on-outgroup ostracism [38,57]. Although this research yielded valuable insights, it does not speak clearly to the effect of the target’s group membership on reactions to ingroup-perpetrated ostracism, which is of central interest in the present studies.
Second, we focus on the effects of ethnic group membership and the case of dominant group members’ ostracism of a disadvantaged ethnic minority group member target in particular–as opposed, for example, to group memberships based on political [38] or computer preferences [56] or those induced experimentally through minimal group paradigms [39,57]. Our focus on this specific case sets our studies apart from past work. Notably, basing group membership on distinctions, such as those instantiated by minimal group paradigms, that are relatively minor, as opposed to distinctions, like ethnic background that are especially salient, essentialized, and affectively charged may account for some of the conflicting or null effects of group membership on observers’ reactions to ostracism. From a social justice perspective, understanding how members of powerful groups react–or fail to react–to the negative treatment of members of historically disadvantaged groups has the potential to illuminate mechanisms through which group power relations are reinforced and perpetuated, or more optimistically, conditions under which dominant group members are more sensitive to injustice.
Our primary dependent measures were participants’ affective reactions to the behavior they observed: Their feelings of guilt, anger, sadness, and fear as well as empathy for the target were assessed, along with general positive affect (as a filler). However, we also considered the implications for individuals’ evaluations of the ostracized target. Of particular interest was whether, as a result of dissonance-related processes, any enhanced reactivity to dominant-on-disadvantaged ostracism might coincide with more negative evaluations of disadvantaged group targets. Specifically, to the extent that individuals experience a social identity threat as a result of witnessing the potentially racist behavior of fellow dominant group members they may react defensively by devaluing or disparaging the disadvantaged target. Indeed, a large body of research indicates that various self as well as social identity threats may motivate defensive reactions ranging from the subtle denial of uniquely human emotions to outgroup members (i.e., infrahumanization [60]) to outright derogation [61–64]. Accordingly, several items assessed participants’ evaluations of the target. Finally, compensatory behavior toward targets and punishment behavior toward perpetrators were assessed as potential downstream consequences of negative affect.
Although most past work examining vicarious ostracism has typically evaluated the extent to which observers themselves feel excluded by assessing, for instance, their need satisfaction (e.g., feelings of belonging and self-esteem), we forwent this approach in the present studies. We were less focused on the intrapersonal consequences of witnessing ostracism and more on its interpersonal implications. For that reason, the affective reactions we measured were those, like anger, guilt, and empathy that might be most likely to motivate observers’ behavioral responses toward the target and perpetrators. Further, observers’ feelings of empathy toward the target might heavily overlap with and operate similarly to their feelings of vicarious exclusion. Our focus on motivating affective responses is aligned with prior research that has sought to evaluate the impact of group membership on observers’ responses to witnessing ostracism rather than assessing reactions irrespective of group membership [16,39].
Although we have outlined theoretical bases for different potential patterns of results for affective reactivity, we refrain from expressing specific predictions here regarding which pattern will be most evident or regarding specific mediational chains. Our intellectual journey to the present theoretical framing of this research has been circuitous, as documented in two pre-registrations that can be accessed at osf.io/9jk3c and osf.io/ng796. Perhaps most notably, we initially had distinct predictions for observing moderate versus severe ostracism and our initial predictions did not focus on specific negative affective states whereas our later ones did. Here we combine together all of the studies that we have conducted to date on this topic, which follow a very similar methodology, and conduct a “mini-meta-analysis” [20] to evaluate the weight of the evidence overall for the reduced or enhanced (or equal) reactivity hypotheses using two-tailed significance tests. In line with our most recent pre-registration (osf.io/ng796), we also test whether effects obtained on behavioral reactions are mediated by affective reactions in an exploratory manner, with the type of reaction tested determined in a data-driven manner by the pattern of results obtained across the various measures.
We now proceed to outline in full the design and procedures of each of the four studies, except that for the dependent measures we only report our focal measures, namely participants’ affective reactions, their feelings of empathy toward the target, as well as their overall impressions of him or her. In addition, we also report results for Study 4 because this study assessed several behavioral responses that were not assessed in the other studies and thus provides a unique opportunity to test indirect effects on behavior. All additional measures not included in the main text are listed in the (S1 File), as are results for Studies 1 to 3. After outlining the method for each of the studies as well as the results for Study 4, we then report the meta-analysis across all four studies and discuss the combined results.
General method
In each study participants were either introductory psychology students or Amazon Mechanical Turk workers, all with a White/European ethnic background who completed an online study of social perception in exchange for partial course credit or 0.50 USD. Studies 1 and 2 sampled only women, whereas Studies 3 and 4 also sampled men. For each individual study sample size was determined in a priori power analyses (α = .05, two-tailed) before any data analysis. Given initial uncertainty about the size of expected effects when we began this program of research, the sample size for Study 1 was selected to provide .95 power to detect the average effect size in social psychology found by Richard et al. ([65]; f = 0.21). In contrast, for Studies 2 through 4, sample size was selected to provide .80 (Study 2) to .85 power (Studies 3 and 4; see pre-registrations for details) to detect a small interaction effect size (f = 0.10–0.11). However, for consistency across studies we report the outcome of sensitivity power analyses for each study indicating the effect size we had .80 power to detect rather than the results of a priori power analyses because the a priori power analyses used to determine the sample sizes for Studies 3 and 4 included accommodations for attrition due to exclusions not implemented in the analyses reported in this paper. A summary of sample characteristics for all studies is provided in Table 1.
In each study participants were randomly assigned to observe the inclusion or ostracism of another ostensible participant in a game of ball-toss by two other ostensible participants, after which their affect, empathy, and impressions were assessed. The game of ball-toss consisted of Cyberball [66], a paradigm widely used to evaluate the effects of directly experiencing ostracism, but which can, by implementing the observer role, be used to examine the effects of witnessing ostracism. In the version of Cyberball we employed, the three players were represented by cartoon avatars and photographs and appeared to toss a ball amongst themselves while the participant watched. Although participants were informed that they and the other participants were randomly assigned to be either players or observers, in reality there were no other participants, all participants were assigned the role of observers, and the behavior of the players was entirely computer-programmed. All studies included a severe ostracism and an inclusion condition. In the severe ostracism condition, participants witnessed the target receive only two out of 30 total tosses (6.67% of the throws) from the other players near the beginning of the game and then never again for the remainder of it. In the inclusion condition, the target, along with the other players, received an equal 10 total tosses (33.33% of the throws). In addition, Studies 2 and 3 also included a moderate ostracism condition in which the target received six out of 30 tosses (20% of the throws) at the start of the game. In all studies, across observed treatment conditions, target type was operationalized by ethnic background and was manipulated through the photos apparently uploaded by the participants ostensibly selected as players. In White/European target games the photo uploaded by the target portrayed an individual with a White/European ethnic background. In Black target games the photo uploaded by the target portrayed an individual with a Black ethnic background. In all games the photos ostensibly uploaded by the perpetrators were of individuals with White/European ethnic backgrounds. These photos were obtained with permission from contacts of the first author and were taken in a candid and realistic “selfie” style to enhance the credibility of the cover story. The photos presented were matched to participants’ gender, but their order and role were not counter-balanced.
As per our pre-registrations, in each study participants were excluded if they: a) did not complete any of the main dependent variables; b) completed the survey, in part or in full, more than once (only duplicate responses beyond the first submitted were excluded); c) indicated a non-White/European ethnic background or a sex other than male or female, or d), were not able to view the manipulation.
However, we deviate from our first pre-registration in two ways. First, our first pre-registration indicated that we would also exclude participants who failed to correctly answer attention-check items or incorrectly answered manipulation check items regarding the ethnicity of the players and the type of observed treatment (indicating whether one person received next to no ball tosses or all players received the ball about equally, or, in Studies 2 and 3, one person received somewhat fewer tosses than the others). Because the exclusion criteria were arguably overly stringent (especially regarding distinguishing between moderate versus severe ostracism) and vulnerable to social desirability motivations (most notably a desire to appear “color-blind” and not notice player ethnicity), we abandoned these going forward to minimize participant attrition and avoid introducing bias into the sample [67].
Likewise, although our first pre-registration indicated a plan to exclude participants who indicated familiarity with or suspicion about the authenticity of Cyberball, we did not exclude on suspicion in any of the analyses reported in the main text. When we evaluated the robustness of the meta-analytic effects to participant suspicion, by coding whether participants indicated suspicion in an open-ended thought-listing task that immediately followed the game of Cyberball in Studies 2 to 4 and then conducting the meta-analysis with these participants excluded, we found that the focal pattern of results obtained across the studies was essentially unaffected. In view of these results, and because we sought to minimize participant attrition, we opted not to exclude potentially suspicious participants in the final analyses reported in the main text. We further note that research revealing that direct experiences with ostracism are still aversive even when people know their treatment was computer generated or scripted [5] suggests that doubts about the authenticity of the manipulation do not fully undermine its effectiveness. Additional details regarding the coding, analyses, and results regarding participant suspicion are provided in the (S1 File).
Study 1
Method
Participants.
A sensitivity power analysis indicated that the sample size of 301 would provide .80 power to detect an effect of f = 0.162 ( = .013).
Procedure and design.
Participants connected to the study website for a study ostensibly examining social perception processes and were instructed that they would engage in an online game of ball-toss (i.e., Cyberball) with three other people who were also currently participating in the study. After being notified that they had been selected for the role of observer, participants were randomly assigned to one cell of a 2 (observed treatment: severe ostracism vs. inclusion) × 2 (target type: White/European vs. Black) factorial design, with each participant observing one game of Cyberball in which either a White or Black target was either included or severely ostracized by other White individuals.
Measures.
Participants reported their current feelings of guilt (guilty, apologetic, ashamed; α = .79), anger (angry), sadness (sad), and positive affect (happy, comfortable, lively, calm, caring; α = .79) on a 9-point scale (1 = definitely do not feel, 9 = definitely do feel). These items were adapted from the Brief Mood Introspection Scale (BMIS [68]) as well as the revised Positive and Negative Affect Schedule (PANAS-X [69]). In all studies where a particular emotion was assessed through multiple items, we formed an overall index of that emotion by averaging across participants’ ratings of the associated items.
Participants’ feelings of empathy toward the target were assessed with a single item embedded among the impression dimensions (see below). This item asked participants to indicate the extent to which they empathized with each player during the game of Cyberball on a 7-point scale (1 = not at all, 7 = very much).
Participants indicated their impressions of all the players in Cyberball, including the target, along several dimensions. Ratings were made separately for each player with the order of the players randomized across participants. Participants indicated how likeable they thought each player was and how much they would trust him or her, as well as how much they identified with and felt similar to each player. Ratings were made on a 7-point scale with higher scores indicating more favorable impressions (e.g., more likeable). In all studies where impressions were assessed with more than two items, a principal component analysis indicated that a single component accounted for most of the variance in participants’ impressions. Consequently, participants’ ratings of liking, trust, identification, and similarity were averaged together to create a single impressions of target index (α = .72). The focal measures employed across all studies are summarized in Table 2. Participants completed the impression items first, then the empathy measure, and then the affect items.
Study 2
Study 3
Method
Participants.
A sensitivity power analysis indicated that the sample size of 1958 would provide .80 power to detect an effect of f = 0.070 ( = .003).
Procedure and design.
The procedure and design of Study 3 was the same as in Study 2 except that photos of men in addition to women were employed in Cyberball.
Measures.
Participants reported their present feelings of guilt (guilt, ashamed; α = .80), anger (hostile, irritable; α = .68), sadness (distressed, upset; α = .78), fear (scared, nervous, jittery, afraid; α = .83), and positive affect (excited, enthusiastic, interested, proud, strong, alert, inspired, determined, attentive, active; α = .91) on a 5-point scale (1 = very slightly or not at all, 5 = extremely). These items were adapted from the Positive and Negative Affect Schedule (PANAS [70]). The order of the items was randomized. Participants’ feelings of empathy toward the target as well as their impressions of the target (α = .86) were measured in the same way as in Studies 1 and 2.
Study 4
Method
Participants.
A sensitivity power analysis indicated that the sample size of 1230 would provide .80 power to detect an effect of f = 0.080 ( = .003; Odds Ratio [OR] = 1.34).
Procedure and design.
The procedure and design of Study 4 was the same as Study 1, except for the addition of male photos in Cyberball.
Measures.
Participants reported their present feelings of guilt (guilty, ashamed, blameworthy, angry at self, disgusted with self, dissatisfied with self; α = .89), anger (angry, annoyed, hostile, indignant, resentful, outraged; α = .88), sadness (sad, blue, downhearted, alone, lonely; α = .87), fear (scared, nervous, jittery, afraid, frightened, shaky; α = .89), and positive affect (happy, joyful, delighted, cheerful, excited, enthusiastic, lively, energetic, proud, strong, confident, bold, daring, fearless; α = .95) on a 5-point scale (1 = very slightly or not at all, 5 = extremely). These items were adapted from the PANAS-X [69] as well as past research in the intergroup domain [52]. The order of the items was randomized.
Participants’ feelings of empathy toward the target were measured by asking participants to indicate the degree to which they currently felt moved, compassionate, warm, soft-hearted, sympathetic, and tender. These items were adapted from the empathy literature [71,72] and were included among the other affect items and rated on the same scale. Participants’ ratings across the items were averaged together (α = .88).
Participants’ impressions of the target were assessed in terms of identification and similarity only. Because there were only two items we did not perform a PCA on impressions of the target in Study 4. These items were highly correlated and so were averaged together to create an impressions of target index (r = .84; α = .91).
In Study 4 we also measured a number of distinct behavioral responses to the witnessed interaction. First, we evaluated whether participants acted to try to alleviate any distress the target may have been experiencing by giving participants the opportunity to select him or her as a partner for a second, ostensible task. That is, participants were able to directly reinclude the ostracized target. Participants were told that there was a short, additional final task for them to complete with another participant. They were asked to select a partner for the task from the three individuals who had previously took part in Cyberball. Participants were further informed that a message would be sent to the individual they selected notifying them that they had been chosen. Photos identical to those shown during Cyberball were displayed to assist participants with their choice. Participants’ selection was coded dichotomously (0 = selected perpetrator; 1 = selected target).
We further assessed whether participants opted to send a message to the target or the perpetrators by giving them the option to send messages to any or none of the Cyberball players (both perpetrators separately and target). Photographs of each of the players were presented alongside a text-box in which participants could enter their message. We coded whether or not participants sent a message to the target (did not send message to target = 0; sent message to target = 1) or one of the perpetrators (did not send message to a perpetrator = 0; sent message to perpetrator = 1).
The content of participants’ open-ended messages was also rated by three coders blind to participants’ condition assignments and the study’s specific objectives. Coders rated how supportive and angry each of the messages were using a 7-point scale (1 = not at all, 7 = very much). The supportiveness of the messages was coded with the following four items: “Does it seem like they trying to comfort this person?”, “how supportive are they toward this person?”, “how much do they seem like they are trying to help this person?”, and “how much do they seem to be sympathizing with this person?”. The anger in the messages was coded with the following four items: “How upset with or angry are they toward this person?”, “does it seem like they are trying to chastise, criticize, or reprimand this person?”, “does it seem like they are blaming this person?”, “how much do they seem to be derogating this person?”. The dimensions on which coders made their ratings of participants’ messages were both adapted from past work examining responses to observed acts of ostracism [73] and supplemented with additional items we created. Coding participants’ messages in this way allowed us to examine the degree to which participants attempted to assuage the target’s distress as well as the extent to which they sought to “call out” the perpetrators’ treatment of the target and perhaps try to persuade them of the harm such actions may cause. Coders’ overall ratings of supportiveness (intraclass correlation coefficient [ICC] = .79 and .78) and anger (both ICCs = .96) for each of the perpetrators were reliable and highly correlated (rs = .84 and .90 for supportiveness and anger respectively) and so were averaged together to a create general index of how supportive (ICC = .92) and angry (ICC = .95) participants’ messages were to both perpetrators combined. Coders’ average ratings of the supportiveness (ICC = .82) and anger (ICC = .94) of participants’ messages to the target were also reliable. In this study participants completed the affect items first, followed by the empathy measure, impressions, partner choice, and messaging opportunity.
Results
Because preliminary analyses of affective and behavioral reactions in Studies 3 and 4 including sex as an additional factor did not yield any consistent effects we do not discuss this factor further (for the results of these analyses see the supporting information, S1 File). Unless otherwise indicated the effects of observed treatment and target type on all dependent variables were examined through factorial analysis of variance (ANOVA). The results of omnibus hypothesis tests for affective reactions, empathy, impressions, and behavioral responses are summarized in Tables 3 and 4. The results of simple effect tests examining the effect of target type within each of the observed treatment conditions are presented in Table 5 for variables where significant interactions emerged.
Affective reactions.
A significant Observed Treatment × Target Type interaction emerged for feelings of anger. After observing severe ostracism, participants felt more anger when the target was Black (M = 1.78, SD = 0.85) as opposed to White (M = 1.52, SD = 0.68), whereas after observing inclusion, they felt less anger when the target was Black (M = 1.27, SD = 0.51), as opposed to White (M = 1.39, SD = 0.62). A significant main effect of observed treatment and a marginally significant main effect of target group membership, both qualified by the aforementioned interaction, were also evident for anger.
The Observed Treatment × Target Type interaction was marginally significant for guilt. Although descriptively the means followed the same pattern as for anger, none of the simple effects of target type were significant in either observed treatment condition.
A marginally significant Observed Treatment × Target Type interaction was also evident for sadness. Although descriptively the means followed the same pattern as for anger, the simple effect of target group membership was not significant in either observed treatment condition. A significant main effect of observed treatment also emerged, such that after observing severe ostracism (M = 1.68, SD = 0.82) relative to inclusion (M = 1.55, SD = 0.78), participants felt more sadness.
A significant Observed Treatment × Target Type interaction also emerged for fear. The simple effect of target group membership was only marginally significant in the inclusion condition, such that participants experienced more fear when the target was White (M = 1.32, SD = 0.60) relative to Black (M = 1.24, SD = 0.52).
For participants’ positive affect a significant main effect of observed treatment was evident, such that after observing severe ostracism (M = 1.89, SD = 0.82) relative to inclusion (M = 2.07, SD = 0.92), participants felt less positive affect.
Feelings of empathy toward target.
A significant main effect of observed treatment emerged on participants’ feelings of empathy toward the target, with participants empathizing more with severely ostracized (M = 2.25, SD = 0.90) as opposed to included targets (M = 2.01, SD = 0.93).
Impressions of target.
A significant main effect of observed treatment was evident on participants’ overall impressions of the target, such that participants reported more positive impressions of severely ostracized (M = 3.76, SD = 1.51) relative to included targets (M = 3.38, SD = 1.43). A significant main effect of target type also emerged, with participants reporting less positive impressions of Black (M = 3.43, SD = 1.44) compared to White targets (M = 3.71, SD = 1.51).
Behavioral responses.
We employed hierarchical logistic regression to determine whether participants’ choice of partner (0 = selected perpetrator; 1 = selected target) for the ostensible final task was impacted by observed treatment (inclusion = 0; severe ostracism = 1) and target type (White/European = 0, Black = 1). We entered observed treatment and target type in Step 1 and added their interaction in Step 2. A significant main effect of observed treatment emerged, such that after observing severe ostracism, relative to observing inclusion, participants were over 7 times more likely to select the target as opposed to one of the perpetrators as their partner for the upcoming task.
With respect to the messaging dependent measure, a significant main effect of observed treatment on the supportiveness of participants’ messages to the target was evident, such that participants sent more positive, supportive messages to the target after they observed him or her being severely ostracized (M = 3.54, SD = 1.22) than after observing him or her being included (M = 2.40, SD = 0.82). Descriptive statistics for all behavioral responses are reported in Table 6.
Examples of supportive messages to the target included: “I don't know what's up with your exclusion from the game but, I disagree with it.” (supportiveness = 3.58) and “Hey there! You did good! but for some reason player 1 and player 3 didn't want to play with you for the rest of the game huh” (supportiveness = 3.58). However, as noted in Table 6, there was a wide range, with some participants’ messages indicating complete unconcern for the target’s mistreatment “Hi there” (supportiveness = 1.50) and others, such as that presented below, a high degree of concern:
“Glad you stuck around while those other two players, those IDIOTS, hogged the ball and played catch solely with each other. Don't let it get to you. It doesn't have anything to do with you, the one person in the group I'd LIKE to play ball with, it has to do with them, their thoughtlessness, their selfishness, etc. Somehow, I think you know all this already. I think you're the only one with any brains or any heart in the group.” (supportiveness = 5.92).
For participants’ messages to the perpetrators, a significant main effect of observed treatment emerged for both how angry and how supportive the messages were, such that participants sent more angry messages to the perpetrators after observing them engage in severe ostracism (M = 3.30, SD = 1.35) than after observing them engage in inclusion (M = 1.31 SD = 0.73), whereas they sent more supportive messages after observing them include (M = 2.37, SD = 0.80) as opposed to severely ostracize the target (M = 1.39, SD = 0.55).
Examples of angry messages to the perpetrators included: “Let player two participate more.” (anger = 3.33) and “Why would you all of a sudden avoid Player 2?” (anger = 4.00). Yet, there was a wide range (see Table 6), with some participants’ messages not revealing any anger toward the perpetrators, “throw in a triangle then reverse” (anger = 1.96), and other messages a high degree of anger, “Why did you feel the need to be an asshole and not pass it to player 2?” (anger = 6.29).
We conducted a hierarchical logistic regression parallel to the one for partner choice to examine the effect of observed treatment and target type on participants’ decision to send a message to the target or perpetrators. This analysis yielded a significant main effect of observed treatment, with participants being over 1.5 times more likely to send targets a message after observing them be severely ostracized as opposed to included. For participants’ decision to send a message to one of the perpetrators, a significant main effect of observed treatment was evident, with participants being almost 1.5 times more likely to send a message to one of the perpetrators after observing them severely ostracize, relative to include, the target.
Mediation analyses.
Although there were significant interactions between observed treatment and target type on feelings of anger and on feelings of fear, there were no parallel direct effects on behavior. To assess the possibility that observing ostracism had indirect effects on behavior via affective reactions [75] we used the PROCESS macro v2.13 (Model 8, with 10,000 bootstrap samples [76]) to test for indirect behavioral effects via anger and fear on participants’ choice of partner, their decision to send a message to the target or one of the perpetrators and the supportiveness and anger of those messages. For all analyses, target type (White/European = 0; Black = 1) served as the predictor and observed treatment (inclusion = 0; severe ostracism = 1) functioned as the moderator.
No significant indirect effects via fear emerged (all 95% bootstrap CIs for the indirect effects included zero for all measures).
Significant indirect effects via anger were present on all measures except anger of participants’ messages to the target. The indirect effects are summarized in Table 7. As participants felt significantly more anger after witnessing the severe ostracism of a Black as opposed to a White target, these results suggest that observing the severe ostracism of a disadvantaged ethnic minority group member had the indirect effect, via increased anger, of increasing participants’ choice of the Black target as their partner for a future task, their likelihood of sending the Black target a message that was more likely to be positive and encouraging, as well as their likelihood of sending one of the perpetrators a message that was more likely to be negative (i.e., angry, chastising) and less positive (i.e., supportive, helpful). Several indirect effects also emerged in the inclusion condition (see Table 7). Because participants felt significantly less anger after witnessing the inclusion of a Black as opposed to a White target, these indirect effects suggest that observing inclusion of a disadvantaged group member had the indirect effect, via reduced anger, of decreasing participants’ choice of the Black target as a partner in a future task and their likelihood of sending a message to the perpetrators or target. Thus, it was only after observing negative treatment in the form of ostracism that increased feelings of anger appeared to motivate observers’ efforts to not only comfort and support disadvantaged group targets, but also confront and challenge the perpetrators of such mistreatment.
Meta-analysis of Studies 1 to 4
We now report the meta-analysis of all four studies. We first outline the approach that we employed to create a combined summary of the results of the individual studies. We then summarize the results of this analysis.
Meta-analytic approach
To meta-analyze results across studies we computed an estimate of the average standardized mean difference in reactions to observing a disadvantaged group target relative to a dominant group target in the form of Cohen’s d across all studies in each observed treatment condition. The weights used to pool individual effect sizes were calculated using the inverse variance method. Given that the studies we conducted examined similar hypotheses using highly similar procedures and measures, we report the average effect sizes computed using a fixed-effects model. The average effect size for each dependent variable that appeared in two or more studies for each condition are presented in Table 8.
Alongside estimates of the average effect size we also evaluated the degree of heterogeneity in effect sizes across studies that is not due to sampling error. We evaluated heterogeneity using both Cochran’s Q and the I2 index [77]. Cochran’s Q assesses whether there is significant heterogeneity in effect sizes, with a significant Q indicating heterogeneity. The I2 index, on the other hand, provides a percentage estimate of the extent of heterogeneity. The I2 index ranges from 0% to 100%, with values of 25%, 50%, and 75% indicating low, moderate, and high heterogeneity. Heterogeneity information accompanies all effect size estimates.
To determine whether the overall effect of target type differed as a function of observed treatment we conducted moderator analyses [20,78]. The overall effect of target type across all observed treatment conditions, the omnibus test of moderation, and the outcome of contrasts comparing each of the specific observed treatment conditions are presented in Table 9.
We also computed the average effect of observed treatment for each dependent variable separately for Black and White targets as well as tested whether the overall effect differed as a function of target type. For these analyses we averaged the moderate and severe ostracism conditions within Studies 2 and 3 before computing the average effect to avoid introducing a dependency in the data by utilizing the inclusion condition twice in effect size computations [78]. The results of these analyses are presented in Tables 10 and 11. Notably, parsing the interaction between observed treatment and target type in this way leads to the same conclusions as focusing on the effect of target type. Consequently, to keep the presentation of results manageable we focus our discussion on the effect of target type. All meta-analytic analyses were conducted using the metafor [79] and meta [80] packages in the R statistical environment [81].
Meta-analytic results
Affective reactions.
Tests of moderation revealed that the combined effect of target type on guilt, anger, and sadness was not consistent across observed treatment conditions (see Table 9). In each case the severe ostracism condition significantly differed from the inclusion condition. Indeed, in the severe ostracism condition participants reported significantly more feelings of guilt, anger, and sadness after observing an interaction with a Black as opposed to White target, whereas in the inclusion condition the opposite tended to be descriptively true, although not statistically significant (see Table 8 for the within-condition simple effects). These effects were small in magnitude. Further, for guilt and anger, the effect of target type in the moderate ostracism condition also significantly differed from its effect in the inclusion condition, whereas for sadness this difference only approached statistical significance. However, although the effect of target type on guilt, anger, and sadness in the moderate ostracism condition descriptively mirrored that obtained in the severe ostracism condition it was not statistically significant.
Overall, across observed treatment conditions participants reported significantly more feelings of guilt and anger after observing an interaction involving a Black relative to a White target, although these main effects were qualified by the aforementioned interactions. The main effect of target type was not significant for sadness. No significant effects emerged for fear or positive affect.
Feelings of empathy toward target.
Tests of moderation indicated that the combined effect of target type on participants’ feelings of empathy toward the target was relatively consistent across observed treatment conditions. Overall, across studies, participants reported significantly greater feelings of empathy toward a Black compared to a White target. The size of this effect was small.
Impressions of target.
Tests of moderation suggested that the influence of target type diverged across observed treatment conditions. The severe ostracism condition significantly differed from both the inclusion and moderate ostracism conditions. In the severe ostracism condition participants reported significantly more negative impressions of Black relative to White targets, whereas in the moderate ostracism and inclusion conditions the effect of target type was reversed, although it was not statistically significant in the inclusion condition. The effect of target type did not differ between the inclusion and moderate ostracism conditions: Only comparisons with severe ostracism yielded significant results.
Discussion
Across four studies involving over 4000 participants from university and community samples we found that after observing a severe act of ostracism perpetrated by fellow dominant group members, White individuals reported significantly more feelings of guilt, anger, and sadness when the target was an ethnic minority as opposed to a White person. These results directly build on research demonstrating that individuals are affected by merely observing another person being ostracized (i.e., “vicarious ostracism”) by highlighting that individuals’ affective reactions to observing ostracism perpetrated by their fellow dominant group members vary according to whether the target is a member of a dominant or disadvantaged ethnic group. As previous work has not directly examined the effect of the target’s ethnic group membership on observers’ reactions, our findings offer novel insights into the factors that moderate how individuals respond to witnessing the ostracism of others.
Overall, the present results are not consistent with the possibility that dominant group members, as a result of an empathy gap [23], bystander apathy [36,37], or motivation to maintain a positive social identity [31], will be less impacted by witnessing ingroup members ostracize a disadvantaged group member as compared to a fellow dominant group member (reduced reactivity hypothesis). In fact, results of the meta-analysis conducted across the four studies indicated that in no condition were participants’ affective reactions significantly weaker when the target was a disadvantaged ethnic minority group member as opposed to a dominant group member. Further, we found little evidence of different levels of empathy for disadvantaged as compared to dominant targets of severe ostracism. For other affective reactions, such as anger, sadness, and guilt, when participants’ reactions across studies were combined together the results indicate that observers’ reactions to dominant-on-disadvantaged ostracism was more negative than reactions to dominant-on-dominant ostracism (enhanced reactivity hypothesis).
Why were observers more affected by dominant-on-disadvantaged ostracism? A key possibility is that individuals found such ostracism more grievous than dominant-on-dominant ostracism because it seemed to reflect discrimination based on the target’s ethnicity. The violation of contemporary norms condemning prejudice and discrimination may have engendered anger and sadness [45]. Moreover, as a result of the salience of their shared dominant group membership with the perpetrators, observers may have subsequently felt guilty by virtue of a social identity threat produced by the perception that members of their ingroup had behaved in a racist manner.
An alternative possibility is that participants’ more negative reactions to the ostracism of a disadvantaged ethnic minority group member merely reflect what they think is the socially expected or acceptable response. However, for several reasons we consider this an unlikely explanation for our findings. Two key considerations are that participants completed all surveys anonymously online (and were reminded of the anonymity and confidentiality of their responses before taking part), and the manipulations were between-subjects such that participants were not aware of the comparison conditions. Another is that we did not include any explicit mention of ethnicity until the very end of the survey, after all the dependent measures had been collected. Both of these aspects of our methodology should have reduced social desirability concerns. Moreover, the focal pattern of results was essentially unchanged when we excluded participants who were suspicious that the purpose of the study was to examine racism and discrimination (see S1 File).
In a further effort to address the plausibility of a social desirability interpretation of our results, we also assessed whether the overall effects were changed in any way when participants’ social desirability concerns were taken into account. Specifically, we performed the meta-analysis with effect sizes from Studies 2 through 4 calculated from the estimated marginal means and their respective standard errors obtained from analyses of covariance (ANCOVA) that included participants’ scores on the impression management subscale of the Balanced Inventory of Desirable Responding Short Form (BIDR-16 [82]). Standard errors were converted to standard deviations for effect size computation. The focal pattern of results obtained across the studies was not meaningfully affected by controlling for participants’ impression management concerns. Additional details about our assessment of social desirability as well as the meta-analytic effects computed with the impression management covariate are provided in the (S1 File).
Overall, the negligible impact on the meta-analytic effects of excluding potentially suspicious participants and controlling for social desirability suggests that the pattern of results that we report is not merely an artifact of demand characteristics. Further, at the same time as we acknowledge that it is difficult to rule out completely the possibility that participants’ responses were influenced by what they considered to be socially desirable reactions, we question whether it is really appropriate to view any influence of perceived social norms in such an anonymous context as this as entirely external to or separate from the person.
It is also notable that, consistent with the possibility that individuals experienced a social identity threat in response to witnessing the potentially prejudiced behavior of fellow dominant group members, we found that White individuals derogated a Black relative to a White target of particularly severe ostracism. This finding is difficult to reconcile with a social desirability account. However, it is consistent with past work indicating that exposure to social norms condoning prejudice can increase the endorsement as well as the expression of prejudice [45,83–85]. Overall, individuals’ impressions of the severely ostracized target may have been more negative in the intergroup case because they felt motivated (because of threat) or justified (because of prevailing norms) in expressing their own prejudices [86]. Disentangling which mechanism might predominate in this context is worthy of deeper investigation in future research.
Although it may be somewhat surprising that observers were more impacted by witnessing the ostracism of a disadvantaged ethnic minority group member as opposed to fellow dominant group members given research documenting that individuals display deficits in empathizing with outgroup targets [87] and tend to overestimate their reaction to prejudiced behavior (e.g., overt racism and homophobia; [35–37]), it is important to recognize that we examined a form of negative treatment distinct from that examined in previous research, one that might tap a more universal experience. Moreover, our results are consistent with theory and research highlighting the power of contemporary norms condemning prejudice and individuals’ concerns with avoiding behaving in a prejudiced manner or appearing prejudiced to others [45,47–49], as well as theory and research on the impact of group membership on observers’ responses to ostracism that stresses that when observers witness the ostracism of a dissimilar person they are inclined to attribute the ostracism to malicious motives such as prejudice or discrimination [16]. Finally, our results are also aligned with research on group-based emotions [88] which indicates that when individuals perceive their ingroup as responsible for harming [51,89] or as unfairly advantaged over an outgroup [52] they are likely to experience negative emotions such as guilt and anger.
Affective reactions as mediators
Although we did not find different direct behavioral implications of observing dominant-on-disadvantaged as compared to dominant-on-dominant ostracism, we did find, consistent with theories that stress that emotions are an important motivator of action [88], several indirect effects through affective reactions. These analyses revealed a role for anger as the most proximal predictor of observers’ behavior. Specifically, observers felt angrier after witnessing a severe act of ostracism perpetrated against a Black relative to a White target and feelings of anger were linked to a number of behavioral responses by observers directed toward helping and supporting the target and confronting the perpetrators. These involved directly reincluding the target by selecting him or her as a partner for a future task, sending the target comforting messages, and sending the perpetrators more negative (i.e., angry, critical) and less positive (i.e., sympathetic, helpful) messages. As indirect effects of anger on behavior were evident in only one study, however, these results should be considered tentative and additional research along these lines would be particularly valuable.
Although our mediation analyses were data-driven and exploratory, the results we obtained are in line with theory and research on group-based emotions that indicates that anger may be a particularly potent motivator of action. For example, in their theoretical integration of the effects of advantaged groups’ prosocial emotions Thomas et al. [90] suggest that anger, particularly when it is experienced by advantaged group members as a function of perceiving their ingroup as responsible for the illegitimate disadvantages faced by other groups, may often motivate behavior designed to change or regulate the advantaged group, such as confronting or protesting the actions of other ingroup members [52,89].
Effect sizes and heterogeneity
It should be acknowledged that although we found significant differences in the levels of guilt, anger and sadness that individuals experienced upon witnessing the ostracism of disadvantaged versus dominant group targets, the size of these differences was small. However, as has been frequently argued, small effects may be meaningful and should not be reflexively trivialized [91]. In the present context, where the ostracism manipulation is quite severe and reactions to it overall fairly strong given that it involves only a briefly observed virtual interaction between strangers (Cohen’s d for the effect of observing ostracism relative to inclusion ranged from 0.17 to 0.49 across studies for guilt, anger, sadness, and empathy), any moderation of the effect in what is, from many perspectives, an unexpected direction, seems notable. At a minimum it would seem clear that there is no evidence for reduced reactivity in this context. It is also important to note that despite being small, the differences we found on participants’ feelings of anger to observing the ostracism of a disadvantaged relative to a dominant group target were linked to increases in behavioral responses that may have meaningful consequences for targets’ well-being as well as for the continued perpetration of ostracism and discrimination.
It is also the case that the levels of the specific types of negative affect we assessed (as reported in Study 4) were not high. Notably, mean levels of positive affect were also low, suggesting that overall participants were not feeling particularly positive or negative. It may be that the online nature of the paradigm granted participants distance from the situation that dampened their reactions. It is also possible that the general framing of the affect questions pushed toward less extreme responses. That is, participants were asked about their current mood, not specifically about how the game of Cyberball they watched made them feel. Regardless, although what is most critical from our perspective, and the focus of our analysis, is the relative difference in reactivity across conditions that vary in terms of the target's status, it is nonetheless important to appreciate that the reactions that we documented were on average not very strong.
Alongside computing estimates of the average effect size across studies, we also assessed the degree of heterogeneity in effects that was not due to sampling error. Across studies the degree of heterogeneity was relatively low (Cochran’s Q was frequently nonsignificant and I2 index was often 0), which would be expected based on the relatively high degree of similarity in the design and methodology of our studies. Isolated instances of significant heterogeneity occurred for positive affect and impressions of the target, both in the inclusion condition, and for empathy toward the target in the moderate ostracism condition. An exception was participants’ feelings of anger, for which Cochran’s Q was significant in both of the ostracism conditions, and marginally significant in the inclusion condition, and the I2 indicated a moderate to high degree of heterogeneity across the conditions (56.90% - 89.20%). As discussed in more detail below, this heterogeneity might be accounted for by methodological differences across Study 4 versus the other studies. Overall, the effect of target type on participants’ affective reactions was generally highly consistent across studies, which suggests that our estimated average effect sizes might be fairly accurate summaries of the individual study effects.
Limitations and future directions
Despite providing novel evidence that dominant group members can have more rather than less extreme affective reactions to witnessing dominant-on-disadvantaged as compared to dominant-on-dominant ostracism, our studies are not without their limitations. In particular, all of our studies utilized the same manipulation of ostracism (i.e., Cyberball). While Cyberball was selected because it is well validated as an effective manipulation of direct and vicarious ostracism, it is unclear whether similar, or conversely, weaker or stronger effects might be obtained with other manipulations. Indeed, research with targets of ostracism suggests that not all ostracism manipulations produce the same effects [92]. Thus, it would be worthwhile for future studies to examine the effects that are evident with alternate ostracism manipulations.
Similarly, in each study we only had a single control condition involving observing inclusion. Although we consider inclusion to be the most appropriate comparison for identifying the effects of ostracism, and indeed inclusion is typically the control condition in ostracism research, the extent to which the effects we obtained are unique to observing ostracism is currently unclear. It is possible that similar effects might be found when members of a dominant group witness fellow dominant group members perpetrate other forms of mistreatment against a disadvantaged group member. Yet, in light of past research documenting that dominant group members often do not react to blatant racism [36,37], our results may suggest that observing ostracism, perhaps because it is such a ubiquitous and relatable experience, might set the stage for enhanced sensitivity to prejudice and injustice. However, this interpretation is tentative and should be evaluated in future research.
Further, throughout all of our studies dominant and disadvantaged group status was defined only by ethnic background and the specific ethnic background of the disadvantaged target did not vary. Although this design decision allowed us to examine how members of a specific dominant ethnic group (i.e., White individuals) react to witnessing the ostracism of members of a particular historically disadvantaged ethnic group (e.g., Black individuals), the generalizability of our findings across other dominant-disadvantaged group relations remains an open question. Indeed, it is possible that dominant group members might be more upset by the ostracism of a Black person relative to members of other ethnic groups. Consistent with this possibility, recent surveys indicate that a majority of Americans view being Black as putting people at a disadvantage in American society, more so than other ethnic groups (e.g., Asian or Native Americans [93]). Dominant group members may also be more sensitive to the ostracism of a disadvantaged group member when disadvantage is defined by ethnic background as opposed to other group memberships (e.g., social class [45]). Relatedly, across studies the dominant versus disadvantaged status of the target overlapped with the ingroup versus outgroup status of the target. Thus, it is not clear that the effects we obtained result from the disadvantaged status or the outgroup status of the target. That some prior research using a minimal group paradigm has found more compensatory positive behavior directed toward excluded ingroup (vs. outgroup) members [39]–which is suggestive of greater negative reactivity to exclusion of ingroup members–perhaps speaks to the importance of disadvantage in driving our effects. However, this conclusion is speculative and awaits further testing. Consequently, to resolve these issues it would be beneficial for future studies to vary the ethnicity of the target as well as define advantage and disadvantage along different dimensions, particularly while holding the ingroup-outgroup status of the target constant.
We also acknowledge the real possibility, especially in view of the number of measures that were administered, that individuals’ responses to some of the items may have been affected by their previous exposure to other questions. Along these lines it is perhaps reassuring that although the affect measures sometimes appeared after the impression and empathy items (Studies 1 to 3) and sometimes before (Study 4), and a thought-listing task was sometimes presented right after the Cyberball game (Studies 2 to 4) and sometimes was not (Study 1), there was no evidence of significant heterogeneity in the effects on sadness or guilt across the studies. At the same time, however, it is also notable that the effects on anger were stronger in Study 4, the only study in which affect was measured first, than in the other studies. Because Study 4 differed from the others in a number of ways (including the specific items in the anger index) it is not possible to draw clear conclusions about the role of order of presentation from the present data. Indeed the possibilities for different types of order effects–both in the present research and more broadly in the literature on observed ostracism and observed discrimination–are extensive and complex. For example, perhaps stronger (or weaker) behavioral effects would have been obtained if the behavioral measures had been administered before the affect items. This issue, which has yet to undergo systematic study in the literature addressing reactions to ostracism, is an important one for future research to address.
Finally, all of our studies took place online. While the anonymity granted to participants by the online context likely helped alleviate social desirability concerns, it is possible that effects might be different if observers encountered the ostracism of an outgroup member perpetrated by their ingroup in person.
Conclusion
Across four large experiments in which individuals perceived that they were engaging in an online interaction, we found that dominant group members reported greater feelings of guilt, sadness, and anger after observing their ingroup perpetrate ostracism against a member of a disadvantaged outgroup as opposed to an ingroup member. Moreover, we found no evidence that dominant group members experienced lower levels of empathy toward disadvantaged group as compared to dominant group targets of ostracism. Although past work has documented deficits in empathizing with outgroups and apathetic responses to overtly racist comments on the part of dominant group members, our results suggest that observing ostracism may be a sufficiently potent and relatable experience that when it occurs across group boundaries it awakens individuals’ sensitivity to injustice and discrimination.
Supporting information
S1 File. Contains additional methodological and analytic details.
https://doi.org/10.1371/journal.pone.0234540.s001
(DOCX)
Acknowledgments
We thank Matthew Quesnel for his technical expertise and help in setting up and troubleshooting several of the study tasks as well as Adefemi Badejo, Gagan Gill, and Victoria Gutscher for their help with data coding.
References
- 1. Baumeister RF, Leary MR. The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychol Bull. 1995;117(3): 497–529. pmid:7777651
- 2. Leary MR, Tambor ES, Terdal SK, Downs DL. Self-esteem as an interpersonal monitor: The sociometer hypothesis. J Pers Soc Psychol. 1995;68(4): 518–530.
- 3. Fenigstein A. Self-consciousness and the overperception of self as a target. J Pers Soc Psychol. 1984;47(4): 860–870.
- 4. Gonsalkorale K, Williams KD. The KKK won’t let me play: Ostracism even by a despised outgroup hurts. Eur J Soc. 2007;37(6): 1176–1186.
- 5. Zadro L, Williams KD, Richardson R. How low can you go? ostracism by a computer is sufficient to lower self-reported levels of belonging, control, self-esteem, and meaningful existence. J Exp Soc Psychol. 2004;40(4): 560–567.
- 6. Zuckerman M, Kernis MH, Guarnera SM, Murphy JF, Rappoport L. The egocentric bias: Seeing oneself as cause and target of others' behavior. J Pers. 1983;51(4): 621–630.
- 7. Zuckerman M, Miserandino M, Bernieri F. Civil inattention exists—in elevators. Pers Soc Psychol Bull. 1983;9(4): 578–586.
- 8. Paolini D, Pagliaro S, Alparone FR, Marotta F, van Beest I. On vicarious ostracism. examining the mediators of observers’ reactions towards the target and the sources of ostracism. Soc Influ. 2017;12(4): 117–127.
- 9. Will G-J, Crone EA, van den Bos W, Güroğlu B. Acting on observed social exclusion: Developmental perspectives on punishment of excluders and compensation of victims. Dev Psychol. 2013;49(12): 2236–2244. pmid:23544860
- 10. Geisen A, Echterhoff G. Do I really feel your pain? Comparing the effects of observed and personal ostracism. Pers Soc Psychol Bull. 2018;44(4): 550–561. pmid:29231078
- 11. Masten CL, Eisenberger NI, Pfeifer JH, Dapretto M. Witnessing peer rejection during early adolescence: Neural correlates of empathy for experiences of social exclusion. Soc Neurosci. 2010;5(5–6): 496–507. pmid:20602283
- 12. van der Meulen M, van Ijzendoorn MH, Crone EA. Neural correlates of prosocial behavior: Compensating social exclusion in a four-player cyberball game. PLoS One. 2016;11(7): e0159045. pmid:27391364
- 13. Güroğlu B, Will GJ, Klapwijk ET. Some bullies are more equal than others: Peer relationships modulate altruistic punishment of bullies after observing ostracism. Int J Dev Sci. 2013;7(1): 13–24.
- 14. Over H, Uskul AK. Culture moderates children’s responses to ostracism situations. J Pers Soc Psychol. 2016;110(5): 710–724. pmid:27176774
- 15. Rudert SC, Ruf S, Greifeneder R. Whom to punish? How observers sanction norm- violating behavior in ostracism situations. Eur J Soc Psychol. 2019;00: 1–16.
- 16. Rudert SC, Sutter D, Corrodi VC, Greifeneder R. Who’s to blame? Dissimilarity as a cue in moral judgments of observed ostracism episodes. J Pers Soc Psychol. 2018;115(1): 31–53. pmid:29911881
- 17. Wesselmann ED, Williams KD, Wirth JH. Ostracizing group members who can (or cannot) control being burdensome. Hum Ethol Bull. 2014;29(2): 82–103.
- 18. Wesselmann ED, Wirth JH, Pryor JB, Reeder GD, Williams KD. When do we ostracize?. Soc Psychol Personal Sci. 2013;4(1): 108–115.
- 19. Wesselmann ED, Wirth JH, Pryor JB, Reeder GD, Williams KD. The role of burden and deviation in ostracizing others. J Soc Psychol. 2015;155(5): 483–496. pmid:26267129
- 20. Goh JX, Hall JA, Rosenthal R. Mini meta-analysis of your own studies: Some arguments on why and a primer on how. Soc Personal Psychol Compass. 2016;10(10): 535–549.
- 21. Wesselmann ED, Bagg D, Williams KD. “I feel your pain”: The effects of observing ostracism on the ostracism detection system. J Exp Soc Psychol. 2009;45(6): 1308–1311.
- 22. Masten CL, Eisenberger NI, Pfeifer JH, Colich NL, Dapretto M. Associations among pubertal development, empathic ability, and neural responses while witnessing peer rejection in adolescence. Child Dev. 2013;84(4): 1338–1354. pmid:23379360
- 23. Gutsell JN, Inzlicht M. Intergroup differences in the sharing of emotive states: Neural evidence of an empathy gap. Soc Cogn Affect Neurosci. 2012;7(5): 596–603. pmid:21705345
- 24. Avenanti A, Sirigu A, Aglioti SM. Racial bias reduces empathic sensorimotor resonance with other-race pain. Curr Biol. 2010;20(11): 1018–1022. pmid:20537539
- 25. Xu X, Zuo X, Wang X, Han S. Do you feel my pain? Racial group membership modulates empathic neural responses. J Neurosci. 2009;29(26): 8525–8529. pmid:19571143
- 26. Bruneau EG, Cikara M, Saxe R. Parochial empathy predicts reduced altruism and the endorsement of passive harm. Soc Psychol Personal Sci. 2017;8(8): 934–942. pmid:29276575
- 27. Hein G, Silani G, Preuschoff K, Batson DC, Singer, T. Neural responses to ingroup and outgroup members’ suffering predict individual differences in costly helping. Neuron. 2010;68(1): 149–160. pmid:20920798
- 28. Lanzetta JT, Englis BG. Expectations of cooperation and competition and their effects on observers’ vicarious emotional responses. J Pers Soc Psychol. 1989;56(4): 543–554.
- 29. Mitchell JP, Macrae CN, Banaji MR. Dissociable medial prefrontal contributions to judgments of similar and dissimilar others. Neuron. 2006;50(4): 655–663. pmid:16701214
- 30. Gutzell JN, Inzlicht M. Empathy constrained: Prejudice predicts reduced mental simulation of actions during observation of outgroups. J Exp Soc Psychol. 2010;46(5): 841–845.
- 31. Tarrant M, Branscombe NR, Warner RH, Weston D. Social identity and perceptions of torture: It’s moral when we do it. J Exp Soc Psychol. 2012;48(2): 513–518.
- 32.
Leach C, Snider N, Iyer A. “Poisoning the consciences of the fortunate”: The experience of relative advantage and support for social equality. In: Walker I, Smith H, editors. Relative deprivation: Specification, development, and integration. Cambridge, UK: Cambridge University Press; 2001. pp. 136–163.
- 33. Miron AM, Branscombe NR, Schmitt MT. Collective guilt as distress over illegitimate intergroup inequality. Group Process Intergroup Relat. 2006;9(2): 163–180.
- 34. Saguy T, Chernyak-Hai L, Andrighetto L, Bryson J. When the powerful feels wronged: The legitimization effects of advantaged group members’ sense of being accused for harboring racial or ethnic biases. Eur J Soc Psychol. 2013;43(4): 292–298.
- 35. Crosby JR, Wilson J. Let’s not, and say we would: Imagined and actual responses to witnessing homophobia. J Homosex. 2015;62(7): 957–970. pmid:25603471
- 36. Karmali F, Kawakami K, Page-Gould E. He said what? Physiological and cognitive responses to imagining and witnessing outgroup racism. J Exp Psychol Gen. 2017;146(8): 1073–1085. pmid:28368199
- 37. Kawakami K, Dunn E, Karmali F, Dovidio JF. Mispredicting affective and behavioral responses to racism. Science. 2009;323(5911): 276–278. pmid:19131633
- 38. Veldhuis TM, Gordijn EH, Veenstra R, Lindenberg S. Vicarious group-based rejection: Creating a potentially dangerous mix of humiliation, powerlessness, and anger. PLoS One. 2014;9(4): e95421. pmid:24759901
- 39. Forbes H, Stark AM, Hopkins SW, Fireman GD. The effects of group membership on college students’ social exclusion of peers and bystander behavior. J Psychol. 2020;154(1): 15–37. pmid:31361210
- 40. Beeney JE, Franklin RG, Levy KN, Adams RB Jr. I feel your pain: Emotional closeness modulates neural responses to empathically experienced rejection. Soc Neurosci. 2011;6(4): 369–376. pmid:21400358
- 41. Meyer ML, Masten CL, Ma Y, Wang C, Shi Z, Eisenberger NI, et al. Empathy for the social suffering of friends and strangers recruits distinct patterns of brain activation. Soc Cogn Affect Neurosci. 2013;8(4): 446–454. pmid:22355182
- 42. Spaans JP, Will GJ, van Hoorn J, Guroglu B. Turning a blind eye? Punishment of friends and unfamiliar peers after observed exclusion in adolescence. J Res Adolesc. 2019;29(2): 508–522. pmid:29700908
- 43. Bernstein MJ, Sacco DF, Young SG, Hugenberg K, Cook E. Being “in” with the in-crowd: The effects of social exclusion and inclusion are enhanced by the perceived essentialism of ingroups and outgroups. Pers Soc Psychol Bull. 2010;36(8): 999–1009. pmid:20693384
- 44. Krill A, Platek SM. In-group and out-group membership mediates anterior cingulate activation to social exclusion. Front Evol Neurosci. 2009;13(1): 1. pmid:19597546
- 45. Crandall CS Eshleman A, O'Brien L. Social norms and the expression and suppression of prejudice: The struggle for internalization. J Pers Soc Psychol. 2002;82(3): 359–378. pmid:11902622.
- 46. Iyer A, Ryan MK. Why do men and women challenge gender discrimination in the workplace? The role of group status and in-group identification in predicting pathways to collective action. J Soc Issues. 2009;65(4): 791–814.
- 47. Shelton JN, Richeson JA, Vorauer JD. Threatened identities and interethnic interactions. Eur Rev Soc Psychol. 2006;17(1): 321–358.
- 48. Sommers SR, Norton MI. Lay theories about white racists: What constitutes racism (and what doesn’t). Group Process Intergroup Relat. 2006;9(1): 117–138.
- 49. Vorauer JD. Dominant group members in intergroup interaction: Safety or vulnerability in numbers?. Pers Soc Psychol Bull. 2003;29(4):49 8–511. pmid:15273003
- 50. Rasinski HM, Geers AL, Czopp AM. “I guess what he said wasn’t that bad”: Dissonance in nonconfronting targets of prejudice. Pers Soc Psychol Bull. 2013;39(7): 856–869. pmid:23813423
- 51. Doosje B, Branscombe NR, Spears R, Manstead ASR. Guilty by association: When one’s group has a negative history. J Pers Soc Psychol. 1998;75(4): 872–886.
- 52. Leach CW, Iyer A, Pedersen A. Anger and guilt about ingroup advantage explain the willingness for political action. Pers Soc Psychol Bull. 2006;32(9): 1232–1245. pmid:16902242
- 53. Rudert SC, Reutner L, Greifeneder R, Walker M. Faced with exclusion: Perceived facial warmth and competence influence moral judgments of social exclusion. J Exp Soc Psychol. 2017;68: 101–112.
- 54. Fiske ST, Cuddy AJC, Glick P, Xu J. A model of (often mixed) stereotype content: Competence and warmth respectively follow from perceived status and competition. J Pers Soc Psychol. 2002;82(6): 878–902. pmid:12051578
- 55. Schaafsma J, Williams KD. Exclusion, intergroup hostility, and religious fundamentalism. J Exp Soc Psychol. 2012;48(4): 829–837.
- 56. Williams KD, Cheung CKT, Choi W. Cyberostracism: Effects of being ignored over the internet. J Pers Soc Psychol. 2000;79(5): 748–762. pmid:11079239
- 57. Arpin SN, Froehlich L, Lantian A, Rudert SC, Stelter M. When “we” or “they” exclude others: Attributing and evaluating ostracism observed in in-groups and out-groups. Compr Results Soc Psychol. 2017;2(2–3): 143–165.
- 58. Fayant MP, Muller D, Hartgerink CHJ, Lantian A. Is ostracism by a despised outgroup really hurtful? A replication and extension of Gonsalkorale and Williams (2007). Soc Psychol. 2014;45(6): 489–494.
- 59. Smith A, Williams KD. R u there? Ostracism by cell phone text messages. Group Dyn. Theory Res. Pract. 2004;8(4): 291–301.
- 60. Castano E, Giner-Sorolla R. Not quite human: Infrahumanization in response to collective responsibility for intergroup killing. J Pers Soc Psychol. 2006;90(5): 804–818. pmid:16737374
- 61. Branscombe NR, Wann DL. Collective self-esteem consequences of outgroup derogation when a valued social identity is on trial. Eur J Soc Psychol. 1994;24(6): 641–657.
- 62. Fein S, Spencer SJ. Prejudice as self-image maintenance: Affirming the self through derogating others. J Pers Soc Psychol. 1997;73(1): 31–44.
- 63. Glasford DE, Dovidio JF, Pratto F. I continue to feel so good about us: In-group identification and the use of social identity—enhancing strategies to reduce intragroup dissonance. Pers Soc Psychol Bull. 2009;35(4): 415–427. pmid:19141621
- 64. McGregor I, Zanna MP, Holmes JG, Spencer SJ. Compensatory conviction in the face of personal uncertainty: Going to extremes and being oneself. J Pers Soc Psychol. 2001;80(3): 472–488. pmid:11300580
- 65. Richard FD, Bond CF Jr, Stokes-Zoota JJ. One hundred years of social psychology quantitatively described. J Pers Soc Psychol. 2003;7(4): 331–363.
- 66.
Williams KD, Yeager DS, Cheung CKT, Choi W. Cyberball (Version 4.0) [Computer software]. West Lafayette, IN: Purdue University; 2012 [cited 2016 Dec 6]. Available from: https://cyberball.wikispaces.com.
- 67. Zhou H, Fishbach A. The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions. J Pers Soc Psychol. 2016;111(4): 493–504. pmid:27295328
- 68. Mayer JD, Gaschke YN. The experience and meta-experience of mood. J Pers Soc Psychol.1988;55(1): 102–111. pmid:3418484
- 69.
Watson D, Clark LA. The PANAS-X: Manual for the Positive and Negative Affect Schedule–Expanded Form. Iowa City, IA: University of Iowa; 1994 [cited 2018 Jan 8]. Available from: https://ir.uiowa.edu/psychology_pubs/11
- 70. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: The PANAS scales. J Pers Soc Psychol.1988;54(6): 1063–1070. pmid:3397865
- 71. Batson D C, Fultz J, Schoenrade PA. Distress and empathy: Two qualitatively distinct vicarious emotions with different motivational consequences. J Pers. 1987;55(1): 19–39. pmid:3572705
- 72. Batson DC, O’Quin K, Fultz J, Vanderplas M, Isen AM. Influence of self-reported distress and empathy on egoistic versus altruistic motivation to help. J Pers Soc Psychol. 1983;45(3): 706–718.
- 73. Masten CL, Morelli SA, Eisenberger NI. An fMRI investigation of empathy for “social pain” and subsequent prosocial behavior. Neuroimage. 2011;55(1): 381–388. pmid:21122817
- 74. Steiger JH. Beyond the F test: Effect size confidence intervals and tests of close fit in the analysis of variance and contrast analysis. Psychol Methods. 2004;9(2): 164–182. pmid:15137887
- 75. Rucker DD, Preacher KJ, Tormala ZL, Petty RE. Mediation analysis in social psychology: Current practices and new recommendations. Soc Pers Psychol Compass. 2011;5(6): 359–371.
- 76.
Hayes AF. Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York: Guilford Press; 2013.
- 77. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta- analyses. BMJ. 2003;327: 557–560. pmid:12958120
- 78.
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to meta-analysis. West Sussex, UK: John Wiley & Sons; 2009.
- 79. Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010;36(3): 1–48.
- 80. Schwarzer G. meta: An R package for meta-analysis. R News. 2007;7(3): 40–45.
- 81.
R Core Team. R: A language and environment for statistical computing. [Computer software]. Vienna: R Foundation for Statistical Computing [cited 2018 May 1]. Available from: http://www.r-project.org/.
- 82. Hart CM, Ritchie TD, Hepper EG, Gebauer JE. The balanced inventory of desirable responding short form (BIDR-16). Sage Open. 2015;5(4): 1–9.
- 83. Blanchard FA, Crandall CS, Brigham JC, Vaughn LA. Condemning and condoning racism: A social context approach to interracial settings. J Appl Psychol. 1994;79(6): 993–997.
- 84. Hsueh M, Yogeeswaran K, Malinen S. “Leave your comment below”: Can biased online comments influence our own prejudicial attitudes and behaviors?. Hum Commun Res. 2015;41(4): 557–576.
- 85. Stangor C, Sechrist GB, Jost JT. Changing racial beliefs by providing consensus information. Pers Soc Psychol Bull. 2001;27(4): 486–496.
- 86. Crandall CS, Eshleman A. A justification-suppression model of the expression and experience of prejudice. Psychol Bull. 2003;129(3): 414–446. pmid:12784937.
- 87. Cikara M, Bruneau EG, Saxe RR. Us and them: Intergroup failures of empathy. Curr Dir Psychol Sci. 2011;20(3): 149–153.
- 88. Mackie DM, Smith ER, Ray DG. Intergroup emotions and intergroup relations. Soc Personal Psychol Compass. 2008;2(5): 1866–1880.
- 89. Iyer A, Schmader T, Lickel B. Why individuals protest the perceived transgressions of their country: The role of anger, shame, and guilt. Pers Soc Psychol Bull. 2007;33(4): 572–587. pmid:17400836
- 90. Thomas E, McGarty C, Mavor KI. Transforming “apathy into movement”: The role of prosocial emotions in motivating action for social change. Pers Soc Psychol Rev. 2009;13(4): 310–333. pmid:19755664
- 91. Prentice DA, Miller DT. When small effects are impressive. Psychol Bull. 1992;112(1): 160–164.
- 92. Godwin A, MacNevin G, Zadro L, Iannuzzelli R, Weston S, Gonsalkorale K, et al. Are all ostracism experiences equal? A comparison of the autobiographical recall, cyberball, and o-cam paradigms. Behav Res Methods. 2014;46(3): 660–667. pmid:24163212
- 93.
Pew Research Center. Race in America 2019. Washington, D.C: Pew Research Center; 2019 [cited 2020 Feb 11]. Available from: https://www.pewsocialtrends.org/2019/04/09/race-in-america-2019/