Exploring the direct and indirect effects of elite influence on public opinion

Political elites both respond to public opinion and influence it. Elite policy messages can shape individual policy attitudes, but the extent to which they do is difficult to measure in a dynamic information environment. Furthermore, policy messages are not absorbed in isolation, but spread through the social networks in which individuals are embedded, and their effects must be evaluated in light of how they spread across social environments. Using a sample of 358 participants across thirty student organizations at a large Midwestern research university, we experimentally investigate how real social groups consume and share elite information when evaluating a relatively unfamiliar policy area. We find a significant, direct effect of elite policy messages on individuals’ policy attitudes. However, we find no evidence that policy attitudes are impacted indirectly by elite messages filtered through individuals’ social networks. Results illustrate the power of elite influence over public opinion.

Participant demographics are available in Table 1

Space Policy Statements:
Leading Question: Regarding The U.S. Commercial Space Launch Competitiveness Act, which encourages private sector investment in space exploration and streamlines commercial space launch activities.

ProGovernment Statements
(a) Opponents of the bill say: "In terms of innovation, the private sector is not suited to long term projects. This is because corporations are based on quarterly reporting. If a project takes 20 years to complete, or even just to show some progress, that project is less likely to receive continual funding. Managers will see money flowing into a program every quarter but with no return on investment. Often, this will lead to a program being cut...This is where government agencies such as DARPA have an advantage over the private sector: they can afford to be more concerned with results than with costs." University says: "the principal benefits from human spaceflight are intangible, but nevertheless substantial." The moon missions of the '60s instilled in Americans a sense of "international prestige and national pride," something Logsdon thinks is best produced by initiatives at the federal level.

ProPrivate Statements
(a) Supporters of the bill say: "This bill will unite law with innovation, allowing the next generation of pioneers to experiment, learn and succeed without being constrained by premature regulatory action...Virtually every space stakeholder group supports this bill.
This bill encourages the private sector to launch rockets, take risks and shoot for the stars." (d) Supporters of the bill say: NASA has tried to get its costs more in line. Since 1996, it has handed over many of the functions related to running the shuttle program to a joint venture operated by Boeing and LockheedMartin called the United Space Alliance. That arrangement reduced the number of contracts on the shuttle program from 29 to one. Total headcount has been reduced to fewer than 2,000 from 2,700 in 1991. It's not inconceivable that a fully privatized operation couldn't trim overhead even more than that.

Appendix 3. Baseline Attitudes Survey & Composite Scores
These are the four questions utilized to establish participants' attitudes in support of private or government investment in space. 1 These questions were also asked in each survey subround. We summarized the strength of each participant's ProGovernment opinion in a single composite score by averaging scores from these four questions. As responses to the first question were on a 5option scale, we computed a rescaled score that ranged from 1 (strongly opposed) to 1 (strongly in favor). 2 The other three questions included 'Yes', 'No', and 'Unsure' response options. For the second proprivate question (Question 4), individuals who answered 'Yes' were given a value of '1', 'Unsure' a '0', and 'No' a '1'. This scale was reversed for the second and third (progovernment investment) questions. Accordingly, a participant that is strongly progovernment investment would have a composite score close to one, and their counterparts that are strongly proprivate investment would have a composite score close to 1. At each round, we compared each participant's composite score to their composite score at baseline, and term this difference the 'shift' in score, which can range from 2 to 2.

Appendix 4. Baseline Political Knowledge Questions
We asked participants the following questions to establish their baseline political knowledge. We have included the answer options in parentheses where the correct answer option is bolded.
"We would now like to ask you a few factual political questions, along with your opinions on several issues relevant to U.S. space policy. Your answers to these questions will be shared with other participants in the study, and you will see other participants' responses as well. You will be presented with several pieces of media information regarding these questions as you go through these pages. Note that you might not see the same information as other people, therefore, some people might have information that you do not have. Also, the people that you share your answers with might not be the same people that are sharing their answers with you. Please click the next button to continue." It could be that learning in social networks is facilitated by specific social contacts and not all of them.
Individuals look to their friends that are the most informed about politics when making decisions about whether to participate and which candidate to support (Huckfeldt, 2001; McClurg, 2006. In the same vein, we expect that policy information shared from the most influential friends will be heeded to a greater extent than information from lessinfluential peers. We characterize these individuals as the most 'popular' group members; those group members with the largest number of people who chose them to be their friends. We call this indegree (as outdegree is three for everyone). To do so, we looked at the number of people who chose each participant to be their friend and identified all participants with the largest indegree as the "most popular". In some networks, there were multiple individuals with the same large indegree. Table Appendix 10 Table 1 depicts the number of most popular friends by popular friend assignment. Taken together, we find no evidence of the effect of a popular friends' assignment on individuals' 17/23 own policy attitudes. Thus, we are more confident in our null results. Still, the social effect might operate via other channels instead, and future work should explore this more directly. We also acknowledge that our randomization works against finding an indirect effect of social information as friends display multiple messages, both for and against space privatization. Perhaps, if individuals were seeded the same messages from their three friends (either all progovernment or all proprivate) then we would see more of an indirect effect. This, however, would substantially complicate the design and randomization. A larger "dosage" (the number, density, or intensity) of information may be needed for social information to overcome official information effects (Neblo, Esterling, & Lazer, 2018).

Appendix 11. Statistical Analysis Details
Parametric statistical models are often used to implement or justify statistical analysis of an experiment. For the present study, we might create a parametric statistical model for the composite score outcome of interest, conditional on the treatment assignment. However, for data with a longitudinal ordinal outcome variable collected from participants connected in a network, such modelbuilding is greatly complicated by the dependencies induced by those network connections and the discreteness of the outcome variable. We avoid the potential pitfalls of model misspecification by using an alternative simple approach with a long history in statistical analysis of experiments, which is to use nonparametric tests of treatment effects based on permutation.
The underlying idea of permutation testing is that if the treatment had no effect on the outcomes, then we would see similar outcomes had the treatment assignments been randomized differently. That is, we can simulate a replication of the current experiment by redoing (shuffling) the random treatment assignment without changing any of the postrandomization variable values, including the observed outcomes. Because we do not tamper with the observed outcomes or the network structure, permutation testing preserves any correlation among the outcomes that would be induced by the existing network. Thus, this type of permutation test is conditional on the network. Note that our design is completely randomized within organization (or network), and so each permutation also only reshuffles assignments within the network. For each pseudoexperiment, we calculate a summary statistic relevant to the treatment effect of interest. Repeating this procedure for many pseudoreplicate 18/23 treatment assignments creates a reference distribution of postrandomization outcome summary statistics assuming the null hypothesis that the treatment had no effect. Any truly observed summary statistic that falls in the tails of this permutationbased reference distribution indicate that the observed data would be unusual to see if the the null hypothesis were true, and thus provide strong evidence against the null hypothesis of no treatment effect. For this permutation scheme to work in the presence of missing data, we must make the permuted assignments for the whole networks (regardless of missing data) and then subset each permuted dataset after it is rejoined with the subsequent data collection to drop the appropriate participants based on their and their friends' missing data patterns.
Thus, these analyses are all conditional on both the observed network structures and missing data patterns. The choice of summary statistic determines the interpretation of the test result, as well as the power to identify any true treatment effect.
This experimental setup resembles a twofactor design (see Table 1 in the main text), where the completely randomized experiment simultaneously assigns both the individual and social factor levels.
The assignment of a single treatment to each participant indirectly assigns the social treatment as well, given the fixed network. As such, we choose summary statistics akin to main effects in oneway ANOVA statistical analysis or interaction effects in twoway ANOVA analyses to focus on various ways in which the treatment assignment may be associated with changes in opinion over the 10round experiment. For each, we use the same set of 5000 permutations to create appropriate permutationbased reference distributions.
First, we examine the null hypothesis that there is no effect of treatment assignment on score shifts from baseline to round 10. For each permuted set of assignments, we calculated the average score shift among those pseudoassigned to progovernment messages and those pseudoassigned to proprivate messages, and then calculated the marginal effect of elite information by taking the difference between these two groupspecific average shifts. Because these calculations do not involve the treatment assignments of friends, we term these measures the marginal effect of elite information, as they reflect the average direct treatment effect regardless of (or averaged over) friend assignments. This summary statistic is similar to the main effect on change score in a oneway ANOVA analysis. In our experimental setup where information from treatment assignments can only be transferred to the social network via direct effects, it may be illogical to consider a case with no direct effect of treatment 19/23 assignment but with indirect effects of social information. Thus, lack of evidence against the null hypothesis would be consistent with no effect of the assigned measures, either directly or indirectly.
On the other hand, strong evidence against the null hypothesis would suggest the presence of some direct effect, but would not rule out the possibility of an indirect social effect as well. As noted above, because our marginal effect of elite information calculation does not involve the network structure, the permutation distribution of this marginal measure reflects the distribution of this measure over all possible individual treatment assignments conditional on keeping the same number of each assignment within each network. That is, the permutation distribution reflects the differences in shifts that might have been observed had individual treatment assignments differed, on average across a possibly artificial distribution of indirect assignments.
To examine indirect social effects, we change from using a summary statistic focused on the direct assignment to a vector of statistics that include network structure via the number of friends assigned to view progovernment messages. For each permuted dataset, we first stratify by the number of friends who were pseudoassigned to view progovernment messages, then calculate the pseudoaverage shift from baseline within each stratum. Finally, we compare strata by subtracting the average pseudoshift from baseline across strata for a total of 6 pairwise comparisons among the 4 strata. This procedure mirrors that used for the marginal effect of elite information, where the individual treatment assignment was used to form only two strata. As before, because we do not consider the direct individual assignment (or pseudoassignment) for each participant in the calculated summary statistics, deviation from the reference distribution would be consistent with significant marginal effects of friend treatment assignments (i.e., social interaction) averaged over a 1:1 distribution of the direct individual treatment assignments. Because we assigned treatments to individuals via a completely randomized experiment with 1:1 randomization, the number of individuals in the strata with 0 or 3 friends assigned to view progovernment messages are expected to be much lower than the other strata.
As such, estimation of effects involving these more sparse strata are likely to suffer from lower power than comparisons that involve only the more prevalent assignment strata.
Finally, we examine interactions between official and social information using the same stratification approach, but now create summary statistics based on eight individualbyfriend treatment assignment strata. Here we no longer average over one aspect of the treatment assignment to 20/23 assess marginal effects, instead choosing an analysis that more closely resembles testing for interaction effects in a twoway ANOVA analysis. Any deviation from a reference distribution comparing two strata with the same level of direct individual treatment assignment would be consistent with a conditional social effect within that direct assignment stratum. Likewise, any deviation from a reference distribution comparing two strata with the same level of social treatment assignment would be consistent with a conditional direct effect within that social assignment stratum.