Americans preferred Syrian refugees who are female, English-speaking, and Christian on the eve of Donald Trump’s election

What types of refugees do Americans prefer for admission into the United States? Scholars have explored the immigrant characteristics that appeal to Americans and the characteristics that Europeans prioritize in asylum-seekers, but we currently do not know which refugee characteristics Americans prefer. We conduct a conjoint experiment on a representative sample of 1800 US adults, manipulating refugee attributes in pairs of Syrian refugee profiles, and ask respondents to rate each refugee’s appeal. Our focus on Syrian refugees in a 2016 survey experiment allows us to speak to the concurrent refugee crisis on the eve of a polarizing election, while also identifying religious discrimination, holding constant the refugee’s national origin. We find that Americans prefer Syrian refugees who are female, high-skilled, English-speaking, and Christian, suggesting they prioritize refugee integration into the U.S. labor and cultural markets. We find that the preference for female refugees is not driven by the desire to exclude Muslim male refugees, casting doubt that American preferences at the time were motivated by security concerns. Finally, we find that anti-Muslim bias in refugee preferences varies in magnitude across key subgroups, though it prevails across all sample demographics.


Randomization strategy
The design is shown in Figure 1 below. We note that the sample will be divided into two. A random half will complete the conjoint+DVs during Wave 1. The other random half will be invited to complete the conjoint+DVs during Wave 2, exactly seven days later. 2 Note that this is not a panel study. Rather, some respondents are randomly assigned to answer our outcome variable questions immediately; others are randomly assigned to respond to these questions seven days later (our aim is to complete data collection before the 2016 general election). We will remove all respondents who complete the survey on or after the 2016 U.S. general elections from main and secondary hypotheses as well as for robustness checks, and note when these respondents for the purpose of exploratory analysis. 2 Note that respondents may or may not complete the survey on the day they receive the invitation.

Treatments
Our instrument will be divided into four sections: the collection of pretreatment covariates, the administration of the treatment, the administration of the conjoints, and the collection of other outcome data. Our control condition will o↵er no treatment, such that it will consist of only three sections: the collection of pretreatment covariates, the administration of the conjoint, and the collection of other outcome data. The treatment conditions will be as follows: 1. Empathy prime: Albertson and Gadarian (2013, 2015a, 2015b inform us that anxiety is a strong emotion driving political attitudes. When politicians instill fear and anxiety in us, they shape our political preferences. We ask whether the inverse is possible: what happens if we instill empathy rather than anxiety? Research has shown that this can be a fruitful approach for reducing prejudice toward transgender people (Broockman and Kalla 2016). Drawing from a real lesson plan designed by the Pulitzer Center (http: //pulitzercenter.org/builder/lesson/16023), this prime will ask respondents the following open-ended questions before administering the conjoint: Imagine that you are a refugee fleeing persecution in a war-torn country.
-What would you take with you, limited only to what you can carry yourself, on your journey?
-Where would you flee to or would you stay in your home country?
-What do you feel would be the biggest challenge for you?
Note that this treatment must be taken as an encouragement treatment: we cannot actually measure whether or not we have instilled empathy.
2. Persuasion: In this experimental condition, we test whether better information about refugees has any e↵ect on refugee inclusion. Indeed, Facchini et al. (2016) show that attitudes toward immigrants in Japan can be changed with a mass education campaign. In the case of the Syrian refugee crisis, one of the most salient arguments we have seen in the US context is how large of a contribution each country has made relative to its size. For example, the U.S. may have originally committed itself to admitting 10,000 Syrian refugees, but this represents only 0.003% of its population. By contrast, Canada originally pledged to host 25,000 Syrian refugees, or 0.07% of its population. Therefore, our persuasion treatment will present the following graph: Then, respondents will be asked if this is new information to them or not, and asked whether they thought the U.S. had committed to relatively more, fewer, or the same number of refugees as the other countries. Finally, respondents will be asked to respond to the following prompt: "In a few sentences, please tell us how this information makes you feel about the US's current level of commitment to resettling refugees."

Survey Components
1. Pre-treatment covariates of respondent: gender, age, race, ethnicity, place of birth, education level, occupation, religion, religiosity, party ID, ethnocentrism, knowledge about US commitment to Syrian refugees. Note that gender, age, education level, religion, and party ID are standard questions in the YouGov panel and will thus not be repeated here. Note also that the ethnocentrism measure is adapted from the original Neuliep and McCroskey (1997) measure, adjusted to 8 items. 3

Treatment:
3 The original measure includes 24 items, each weighted equally. For eight items, each weighted equally with 5 points each, we will consider anyone scoring above 27 as high ethnocentric and anyone scoring below 17 as low ethnocentric, which is directly scaled from the original scoring scheme.

Main outcomes of interest
Following the studies on immigrant exclusion and Facchini et al.'s (2016) work on immigrant inclusion, we consider whether there is room to shift respondent opposition (or less favor) towards Syrian refugees (Y 2, Y 3) as well as willingness to send an anonymous letter to the next President of the U.S. in support of resettling refugees (Y 6). Specifically, we explore whether there are e↵ects of guiding respondents through an empathy exercise (T1) or a persuasive argument on the proportional burden of refugees the United States has committed to in comparison to similar Western countries (T2) on Y 2, Y 3 and Y 6. For analysis purposes we can combine Y 2 and Y 3 as two di↵erent observations on the same outcome. To that e↵ect, we refer to Y 2/Y 3 as simply Y 2 continuing forward unless otherwise specified. We expect there to be positive e↵ects of both treatment types on Y 2/Y 3 and Y 6, and for these e↵ects to persist (we hypothesize treatment e↵ects to be captured in Wave 2). We also consider in our primary hypothesis section whether there is a heterogeneous treatment e↵ect that is positive for both treatments with a Muslim refugee profile on the admittance of the refugee (Y 2).
Below are our main outcomes of interest: Y2 On a scale from 1 to 7, where 1 indicates the United States should absolutely not admit the refugee and 7 indicates that the United States should definitely admit the refugee, how would you rate Refugee 1/2? Y 2 and Y 3 are numeric variables taking values from 1 to 7.
Y6 Behavioral question: send an anonymous letter to the next President of the U.S. in support of resettling refugees Y 6 is a binary variable where sending an anonymous letter is coded as 1 (This question asks the respondent if s/he is willing to send a comment in a letter we will compile and send to the next President. If the respondent answers "yes", and then fills out a sensical comment, s/he is coded as "1") and not sending is coded as 0.

Main predictors of interest
The main independent variables are Treatment 1 (Empathy treatment) and Treatment 2 (Persuasion treatment). We expect both types of treatments to have positive e↵ects on our main outcomes of interest, Y 2 and Y 6 both in Wave 1 and in Wave 2. 4 We also hypothesize a possible positive heterogeneous treatment e↵ect between the treatments and whether the refugee profile presented in the conjoint indicates that the refugee is Muslim on Y 2. We code whether the refugee is Muslim or Christian as the variable 'M', where M = 1 if the refugee is Muslim and 0 otherwise. We can consider such a positive heterogeneous treatment e↵ect with a 'Muslim' refugee profile of particular substantive importance in light of the negative biases held against Muslims by Americans ( (Panagopoulos, 2006), (Kalkan, Layman and Uslaner, 2009), (Savelkoul et al., 2011)), as documenting such an e↵ect would provide supportive evidence that such negative biases can be ameliorated through empathy exercises and/or persuasive arguments.

Hypotheses
We present our hypotheses below. A full mapping of hypotheses, outcomes, and specifications can be found in Table 1.

Primary hypotheses
We test the di↵erences in outcomes (Y2 and Y6) between each of the treatments and our control group. For the heterogeneous treatment e↵ect of the Muslim profile and either T1 or T2, we specify a linear model but plan on conducting diagnostic tests suggested in  to determine whether our interaction specification satisfies linearity and common support assumptions. If not, we use Hainmueller et al.'s proposed kernel estimator.
For all hypotheses, we report a main specification without controls and one with the following controls, captured by the vector X: gender, age, US born, education level, religion, party ID, and ethnocentrism. We follow Lin (2013) and use the demeaning for X construction as well as interactions with the treatment to control for covariates (e.g. Y = T + (X X ) + T ⇤ (X X )). For the purposes of clarity below, however, we simply present the regressions in the conventional Y = T formulation in the estimating equations to follow. We will also run the same specifications with de-meaned covariates and their interactions with the treatments but do not present the equations here. Again, for Y 2 outcomes we cluster errors by respondent.
H1a (Empathy e↵ect: score) Respondents who receive the empathy prime will give higher admission scores (7 point scale) than those in the control group. That is, 1 > 0 in the below estimating equation: where ↵ 0 is the intercept.
H1b (Persuasion e↵ect: score) Respondents who receive the persuasion treatment will give higher admission scores than those in the control group. That is, 1 > 0 in the below estimating equation: where ↵ 0 is the intercept.
H2a (Empathy e↵ect: letter) Respondents receiving the empathy prime will be more likely than those in the control group to send an anonymous letter to the next President of the U.S. in support of resettling refugees. That is, 1 > 0 in the below estimating equation: where ↵ 0 is the intercept.
H2b (Persuasion e↵ect: letter) Respondents who receive the persuasion treatment will be more likely than those in the control group to send an anonymous letter to the next President of the U.S. in support of resettling refugees. That is, 1 > 0 in the below estimating equation: where ↵ 0 is the intercept.
H3a (Long term empathy e↵ect: score) The empathy prime e↵ect on improving scores given to refugee profiles may degrade over time, but will persist. That is, 1 > 0 in the below estimating equation: where ↵ 0 is the intercept, Y 2 0 is the measurement of Y 2 at Wave 2.
H3b (Long term persuasion e↵ect: score) The persuasion e↵ect on improving scores given to refugee profiles may degrade over time, but will persist. That is, 1 > 0 in the below estimating equation: H4a (Long term empathy e↵ect: letter) The empathy prime e↵ect on sending an anonymous letter to the next President of the U.S. in support of resettling refugees may degrade over time, but will persist. That is, 1 > 0 in the below estimating equation: where ↵ 0 is the intercept, Y 6 0 is the measurement of Y 6 at Wave 2.
H4b (Long term persuasion e↵ect: letter) The persuasion e↵ect on sending an anonymous letter to the next President of the U.S. in support of resettling refugees may degrade over time, but will persist. That is, 1 > 0 in the below estimating equation: where ↵ 0 is the intercept, Y 6 0 is the measurement of Y 6 at Wave 2.

Secondary hypotheses
As a set of secondary hypotheses, we examine heterogeneous treatment e↵ects on Y 2 for those who view a Muslim profile. For behavioral outcome Y 6 we consider whether the number of Muslim profiles seen by respondents correlates with a larger Muslim "penalty". For this, we construct a variable C consisting of the number of Muslim profiles out of a total of 6 profiles (three sets of two profiles) presented to the respondent, where C 2 [0, 6].
H5a (Heterogeneous e↵ect of empathy and Muslim: score) The empathy prime interacted with a Muslim refugee profile will lead to higher admission scores on average than no treatment interacted with Muslim refugee profiles. That is, 3 > 0 in the below estimating equation: where ↵ 0 is the intercept, and M is a binary variable for whether the refugee profile is Muslim.
H5b (Heterogeneous e↵ect of persuasion and Muslim: score) The persuasion treatment will improve scores for Muslim refugee profiles and will be correlated with higher admission scores than Muslim refugee profiles in the control. That is, 3 > 0 in the below estimating equation: where ↵ 0 is the intercept, M is a binary variable for whether the refugee profile is Muslim.
H6a (Heterogeneous e↵ect of empathy and Muslim: letter) The empathy prime combined with seeing Muslim refugee profiles will be correlated with a higher likelihood of sending an anonymous letter to the next President of the U.S. in support of resettling refugees, controlling for the number of Muslim refugee profiles seen. That is, 3 > 0 in the below estimating equation: where ↵ 0 is the intercept, C is the number of Muslim refugee profiles presented.
H6b (Heterogeneous e↵ect of persuasion and Muslim: letter) The persuasion prime combined with seeing Muslim refugee profiles will be correlated with a higher likelihood of sending an anonymous letter to the next President of the U.S. in support of resettling refugees, controlling for the number of Muslim refugee profiles seen. That is, 3 > 0 in the below estimating equation: where ↵ 0 is the intercept, C is the number of Muslim refugee profiles presented.

Additional hypotheses: robustness checks and verifications
Of secondary importance, we explore several further questions. First, we will verify whether there is evidence in support of what has already been well-established in the American politics literature -a Muslim "penalty" in inclusionary attitudes of Americans for an additional set of outcomes. We cluster errors by respondent for outcomes on Y 1, Y 2 and Y 3.
H7a Given a choice between choosing a Christian refugee profile versus a Muslim refugee profile, respondents are on average more likely to prefer the Christian refugee profile. That is, 1 < 0 in the below estimating equation: H7b Respondents will give lower admission scores on average to Muslim refugees relative to Christian refugees ("Muslim penalty"). That is, 1 < 0 in the below estimating equation: H7c The likelihood of respondents sending an anonymous letter to the White House is negatively correlated with the number of Muslim refugee profiles presented to them.
That is, 1 < 0 in the below estimating equation: Next, we consider long-term heterogeneous treatment e↵ects of viewing Muslim profiles.
H8a The heterogeneous e↵ect of the empathy treatment with the ratings of a Muslim refugee profile compared to a Christian refugee profile may degrade in magnitude over time but will persist. 3 > 0 in the below estimating equation: where Y 0 2 is the measurement of Y 2 at Wave 2.
H8b The heterogeneous e↵ect of the persuasion treatment with the ratings of a Muslim refugee profile compared to a Christian refugee profile may degrade in magnitude over time but will persist. 3 > 0 in the below estimating equation: where Y 0 2 is the measurement of Y 2 at Wave 2.
H9a The heterogeneous e↵ect of the empathy treatment with seeing Muslim refugee profiles on sending an anonymous letter to the White House may degrade in magnitude over time but will persist. 3 > 0 in the below estimating equation: where Y 0 6 is the measurement of Y 6 at Wave 2.
H9b The heterogeneous e↵ect of the persuasion treatment with seeing Muslim refugee profiles on sending an anonymous letter to the White House may degrade in magnitude over time but will persist. 3 > 0 in the below estimating equation: where Y 0 6 is the measurement of Y 6 at Wave 2.
As exploratory analysis, we consider whether there might be a heterogeneous treatment e↵ect of our empathy/persuasion treatments with whether a refugee profile is Muslim on Y 4, the inclusionary attitude for the U.S. taking in Syrian refugees who pass a government security screening. We use our constructed C variable for these exploratory tests.
H10a (Heterogeneous e↵ect of empathy and Muslim: screening) The empathy prime combined with seeing Muslim refugee profiles will be correlated with a higher likelihood of favoring the U.S. taking in Syrian refugees who pass a government security screening relative to the control, controlling for the number of Muslim refugee profiles seen. That is, 3 > 0 in the below estimating equation: where ↵ 0 is the intercept, C is the number of Muslim refugee profiles presented.
H10b (Heterogeneous e↵ect of persuasion and Muslim: screening) The persuasion prime combined with seeing Muslim refugees profile will be correlated with a higher likelihood of favoring the U.S. taking in Syrian refugees who pass a government security screening than in the control, controlling for the number of Muslim refugee profiles seen. That is, 3 > 0 in the below estimating equation: where ↵ 0 is the intercept, C is the number of Muslim refugee profiles presented.
H11a The heterogeneous e↵ect of the empathy treatment and seeing Muslim refugee profiles (compared to a Christian refugee profile) on favoring the U.S. taking in Syrian refugees who pass a government security screening may degrade in magnitude over time but will persist. 3 > 0 in the below estimating equation: where Y 0 4 is the measurement of Y 4 at Wave 2.
H11b The heterogeneous e↵ect of the persuasion treatment with and seeing a Muslim refugee profile compared to a Christian refugee profile on the likelihood of favoring the U.S. taking in Syrian refugees who pass a government security screening may degrade in magnitude over time but will persist. 3 > 0 in the below estimating equation: where Y 0 4 is the measurement of Y 4 at Wave 2.
As a robustness check for our primary outcome Y 2, we will check to see whether the empathy and persuasion treatments have similar positive e↵ects on outcome Y 4 that should reasonably have some correlations with Y 2. We will also check for whether there is an overall reduction in discrimination on any basis in refugee pair profiles (reported di↵erences in scores assigned to Y 2 and Y 3).
As we simply pooled responses to Y 2 and Y 3 amongst all respondents in the primary hypotheses section, we will also run a robustness check on the relevant estimating equations with respondent-level grouped errors (not detailed below).

H12a
The empathy prime will significantly increase the respondent's reported inclusionary attitude for the U.S. taking in Syrian refugees who pass a government security screening. That is, 1 > 0 in the below estimating equation: H12b The persuasion treatment will significantly increase the respondent's reported inclusionary attitude for the U.S. taking in Syrian refugees who pass a government security screening. That is, 1 > 0 in the below estimating equation: H13a The empathy prime will significantly decrease the magnitude of the average di↵erence in admission scores across refugee pairs (i.e. reduce discrimination on any basis).
where Y 2,3 is the absolute di↵erence between Y 2 and Y 3 in a given pair of refugee profiles: |Y 2 Y 3|.

H13b
The persuasion treatment will significantly decrease the magnitude of the average difference in admission scores across refugee pairs (i.e. reduce discrimination on any basis).
where Y 2,3 is the absolute di↵erence between Y 2 and Y 3 in a given pair of refugee profiles: |Y 2 Y 3|.
Finally, although the study is not powered or designed to specifically detect di↵erences between the empathy and persuasion treatment e↵ects, as the literature does not have a clear comparison between the two types of treatments as of this writing, we consider this exercise an exploratory analysis to shed some light on such a comparison.
H14a Respondents who receive the empathy prime will have significantly di↵erent point scores reported for refugee profiles than those in the persuasion group. That is, 1 6 = 0 in the below estimating equation: where T 1,2 is a binary variable that takes the value 1 when the respondent is in the empathy group and 0 when the respondent is in the persuasion group.
H14b Respondents who receive the empathy prime will have a significantly di↵erent preference for the U.S. taking in Syrian refugees who pass a government security screening than respondents in the persuasion group. That is, 1 6 = 0 in the below estimating equation: where T 1,2 is a binary variable that takes the value 1 when the respondent is in the empathy group and 0 when the respondent is in the persuasion group.
H14c The empathy prime group will have a significantly di↵erent preference compared to the persuasion group toward sending an anonymous letter to the White House. 1 6 = 0 in the below estimating equation: where T 1,2 is a binary variable that takes the value 1 when the respondent is in the empathy group and 0 when the respondent is in the persuasion group.
H15a The e↵ects of the empathy treatment on assigning higher point scores to refugee profiles will degrade significantly di↵erently than the persuasion treatment over time. 1 6 = 2 in the below estimating equation: where Y 0 2 is measurement of Y 2 in Wave 2.
H15b The e↵ects of the empathy treatment on favoring or opposing the U.S. taking in Syrian refugees will degrade significantly di↵erently than the persuasion treatment over time.
1 6 = 2 in the below estimating equation: where Y 0 4 is measurement of Y 4 in Wave 2.
H15c The e↵ects of the empathy treatment on sending an anonymous letter to the White House will degrade significantly di↵erently than the persuasion treatment over time.
1 6 = 2 in the below estimating equation: where Y 0 6 is measurement of Y 6 in Wave 2.
If respondents provide enough variation in text for their answers to Y 6, we will conduct text analysis for major themes present in the texts. In particular, we will explore patterns and types of themes in what respondents would take with them if they were refugees and how they feel about the US commitment to Syrian refugees. We will also use the quantity of (non-nonsensical) text as another measure of compliance.
Finally, we will conduct a manipulation check by analyzing responses for Y 5 to see if there is evidence of social desirability bias.

Power calculation
In the Facchini, Margalit and Nakata (2016) study, the authors found that the information treatment had between a 12 and 21 percentage point increase in approving acceptance of an immigrant; over a longer period of time, this e↵ect tended to halve. Our positive persuasion treatment is most similar to the Facchini et al. intervention and so we reflect our estimated treatment e↵ect of the positive persuasion treatment accordingly. Our power calculations are designed to capture our primary hypotheses.
We consider a scenario where the short term positive persuasion treatment size is a 9.5% increase in the outcome variable, the short term empathy treatment size is 19.5%, and long term treatment e↵ects of both treatments are merely half of the short term e↵ect sizes. We also test two heterogeneous treatment e↵ects, the persuasion treatment and a Muslim refugee profile interacted together, as well as an empathy treatment and Muslim refugee profile interacted together. We assume the Muslim e↵ect to be -10, which is consistent with the di↵erence between the average American's reported feelings for Muslims (in a thermometer measure) compared to the average American's reported feelings for an average American, as reported in Kalkan et al. (2009). We set the SD on the outcome to 35 in order to be conservative in our power calculations and require a power of 80% (this is also in the realm of the SD of the e↵ect size uncovered by Hainmueller and Hopkins (2015)). For these tests, we would need a total sample size of N = 5, 300 to detect treatment e↵ects (compared to the control arm) for each treatment in both the short and long terms. See the appendix for the full R code used to generate these power calculations. Figure 2 below illustrates the changes in our sample size needs as we shift some of our assumptions:  Figure 2: Power calculations. The x-axis depicts the sample size (n), while the y-axis represents the power achieved from 0 to 1. The green horizontal line is power at 0.8. Black scatter points are power calculations made with SD of the outcome variable set at 35, while red ones are power calculations made with SD set at 30; all other parameters were set to be the same (assumed treatment e↵ects and heterogeneous treatment e↵ects). The required n-size for power calculations made with SD set at 35 is 4,300, while the required n-size for power calculations made with SD=30 is 5,500.

Multiple comparisons
In order to address the problem of multiple comparisons, we restrict the number of primary hypotheses. We restrict ourselves to two major dependent variables (Y 1, Y 6), two treatments (T 1, T 2) for short and long terms, for a total of eight hypotheses. We also look at a set of secondary hypotheses for heterogeneous tests (T 1 ⇤ X, T 2 ⇤ X). For this, we have a total of four hypotheses. The outcome variables are not likely to be independent, so we take into account this dependency and use a rule of thumb ↵ = 1 (0.05) 4 Survey Instrument S1 Which of these statements best describes you? SC0 Randomization of SC sample into equal halves. One half gets the below questions right away. The other half gets the below questions 1 week later.
SC1 Imagine that you are an o cial making decisions about which refugees to let into the United States. On the next few pages, you will see descriptions of two refugees from Syria and then you will be asked a set of questions about them. Please read the descriptions of the refugees carefully.  SC8 (If yes above) Please use the space below to express your support of refugees to the next president of the United States. The research team will compile these entries and submit them in a letter to the president after he or she is inaugurated in 2017. Your response will be completely anonymous. [Open ended/Decline] SE0 Randomization of SE sample into equal halves. One half gets all the below questions right away. The other half gets SE1 right away, and the rest 1 week later.
SE1 Imagine that you are a refugee fleeing persecution in a war-torn country. In the pages that follow, you will be asked a set of questions about how you imagine this experience would be like for you. You will have space to provide written responses.
Please answer each question with as much detail as you can.