Item issue salience in the news media

Our first hypothesis on item issue characteristics refers to the role of media issue salience in distributive opinion polarization. From the psychological theory of directly motivated reasoning [16], we can hypothesize that mere exposure to information on an issue through the media is likely to lead to distributive opinion polarization. Directly motivated reasoning refers to the (unconscious) strategy of people to seek out information that reinforces their preferences (i.e., confirmation bias), denigrate attitudinal incongruent arguments (i.e., disconfirmation bias), and evaluate information supporting their prior attitudes as stronger and more compelling than counter attitudinal information (i.e., prior attitude effect) [17] (p. 757). Directional motivational reasoning implies that processing additional information on an issue is likely to sharpen citizens´ prior beliefs and attitudes on the particular issue, which in turn increases attitudinal polarization [17]. Empirical studies have indeed shown that directly motivated reasoning leads citizens to endorse stronger opinions (i.e., be more polarized) on an issue after having been exposed to new information on this issue. This effect appears in particular among those who have strong prior opinion on the respective issue and those who are more politically knowledgeable, as the former have affective links to the issue and the latter possess more ammunition to counter information disconfirming their prior beliefs [17, 18].

The (unconscious) activation of directly motivated reasoning is independent of the content of the information to which one is exposed (i.e., whether the information content confirms or disconfirms prior beliefs) [17]. Thus, the mere exposure to media news on an issue is likely to induce directly motivated reasoning among citizens (in particular, those with strong prior beliefs and those more politically knowledgeable) who would then hold more polarized opinions on the respective issue. Indeed, Wojcieszak et al. [18] showed that citizens in the Netherlands who were both fervent supporters and opponents of the EU held more polarized opinions after being exposed to media news about the EU. Therefore, we expect items with a high issue salience in the media to be more polarizing. By media issue salience, we refer to the relative coverage the news media allocates to a given issue [19].

1. H1: The higher the media salience of an item’s issue, the more polarizing the item will be.

Items on cultural issues

Our second hypothesis on item issue characteristics focuses on the extent to which items measuring cultural issues polarize more than items measuring other issue domains. Literature on political cleavage and value change suggests that citizens are increasingly divided on cultural-related issues such as gender, LGTBQI+, or immigration [e.g., 20–23]. Most prominently, Norris and Inglehart [24] develop this thesis in their book “The cultural backlash”. Accordingly, the long-term societal value shift toward cultural liberalization (or “postmaterialism” in Inglehart´s terminology) that started in the aftermath of WWII in advanced democracies has triggered an authoritarian reflex among those who feel most threatened by these changes as they fear losing their majority status. Those who feel threatened by this societal value shift are likely to react by endorsing more authoritarian attitudes on cultural issues, such as the rejection of diverse lifestyles of groups that are perceived as violating conventional norms and traditional customs, including anti-LGTBQI+ and anti-immigrant attitudes or opposition to gender emancipatory roles and norms [24]. According to the cultural backlash theory, the value shift towards cultural liberalization is expected to have deepened opinion polarization on cultural issues in many advanced democracies.

Empirical studies on the evolution of a cultural backlash in the European context were inconclusive: opinion divide on cultural issues along generational or educational lines (the two key socio-demographic characteristics mentioned by [24]) has not increased in Western Europe [9, 25]. Nevertheless, and more importantly for our purpose, the political cleavage literature highlights that cultural-related issues have become more salient and conflictual than other issues -such as socio-economic issues- among the Western European population [e.g., 20, 23]. By extension, whether an item measures a cultural issue could affect its polarizing level. We, therefore, hypothesise that:

1. H2: Items on cultural issues are more likely to polarize than items on non-cultural issues.

Item targeting individual costs and benefits

Turning now to item formulation characteristics, we introduce the distinction between items measuring individual costs and items measuring individual benefits. Prospect theory [26, 27] provides a theoretical framework for explaining why items implying individual costs might be more polarizing than items implying individual benefits. According to Kahneman and Tversky [27], people perceive the outcome of a decision in terms of gains and losses defined with a reference point (usually the current state) rather than in terms of the final stage. Indeed, “the Humans described by prospect theory are guided by the immediate emotional impact of gains and losses, not by long-term prospects of wealth and global utility” [26]. This tendency helps us understand how respondents tend to answer attitudinal items on implementing a policy or political measure: respondents tend to evaluate the kind of changes the policy implementation would imply in terms of losses or gains to them personally.

Moreover, Kahneman and Tversky demonstrated that people put more weight on losses than on gains: “The aggravation that one experiences in losing a sum of money appears to be greater than the pleasure associated with gaining the same amount” [27]. This cognitive mechanism makes individuals loss aversive in their behaviour by preferring the avoidance of losses over the acquisition of equivalent gains. In addition and more critical to our purpose, it implies that individuals respond more to losses than gains ([26], see also [28] for empirical evidence). Pierson [29] pointed to the prospect theory’s relevance to understand the imbalance between citizens’ reactions to social policy cuts and welfare expansion. Moreover, this tendency of reacting more strongly to losses than to gains might lead to a more extensive opinion polarization toward policies or a political matter implying individual costs (such as the implementation of a new tax) than toward policies or a political matter implying individual benefits (such as the allocation of a financial bonus). Indeed, a stronger reaction to attitudinal items on individual costs would concretely manifest in a larger tendency to use the extreme answer categories when answering attitudinal items measuring individual costs. This, in turn, leads to larger opinion polarization at the aggregate level.

Furthermore, Kahneman and Tversky [27] expect this loss aversion mechanism to apply to decisions about both financial and non-financial attributes. Accordingly, respondents are more likely to react more strongly to items measuring attitudes towards financial and non-financial individual costs than to items measuring financial and non-financial individual benefits. An example of a non-financial individual cost item from our sample concerns the implementation of a highway speed limit. An example of a non-financial individual benefit item from our sample is lifting the COVID-test obligation to attend schools during the pandemic. In sum, prospect theory enables us to derive the following hypothesis:

1. H3: Items measuring attitudes towards financial and non-financial individual costs polarize to a larger extent than items measuring attitudes toward financial and non-financial individual benefits.

Items targeting minorities

Another item formulation characteristic that might be related to distributive polarization is the distinction between minority and non-minority targets. The literature on identity politics can help us use this distinction to draw a further hypothesis on the potential polarizing power of minority targets. Identity politics refers to policies aiming to improve the position and status or strengthen the societal recognition of individuals because of their membership in an underrepresented group. Identity politics involve deliberate group boundary-making that implies excluding the “Other”. Recently, several scholars drew attention to some societal drawbacks of identity politics. For instance, Fukuyama [30] pointed to the risk of identity politics turning into excessive political particularism: identity politics implies focusing on ever smaller underrepresented groups while neglecting to build inclusive solidarity around large collectivities. In a similar vein, Lilla [31] argues that identity politics is divisive as those who are not targeted by identity politics feel excluded (such as the white male working class) (see also [32]).

Some European scholars also discussed the potential drawbacks of identity politics as it can trigger fragmentation and hamper social solidarity or a shared sense of collective purpose [e.g., 33]. These criticisms shed light on the risk of societal fragmentation and lack of inclusive solidarity around large collectives resulting from a (policy) focus on underrepresented or minority groups. Furthermore, despite their objective of belonging to a privileged group, members of the majority can feel excluded if they perceive their group as less valued or less recognized [34]. Policies targeting exclusively minority groups can enhance such feelings of exclusion: policies targeting minorities while excluding the majority group are significantly more divisive than all-inclusive policies [35, 36]. By extension, we expect items targeting a minority group to be more polarizing than other items. Items targeting a minority group are more likely to be perceived as securing or implementing particularistic interests, leading to a more affective and stronger response and, thus, polarization. An item example from our sample targeting a minority is “Are you in favour of maintaining dual citizenship in Germany?”. This leads us to formulate the following hypothesis:

1. H4: Items targeting a minority group are more likely to polarize.

Concrete vs. abstract item formulation

Our third hypothesis on item formulation characteristics refers to the item level of abstraction. The literature suggests that people often respond more strongly to objects or events that are more immediate in both time and location and are actual rather than hypothetical (see, for instance [37]).

First, a study by Dochow-Sondershaus and Teney [10] points to the fact that immigration-related attitudinal items measuring particular policies–such as restriction on immigration flow–were more likely to be divisive across occupation classes in Germany. By contrast, items measuring broader principles on immigrants, such as the extent to which immigration is good or bad for the economy–a typical anti-immigrant survey item included in various international social surveys–are more likely to provoke consensual opinion among Germans regardless of their occupational classes. Thus, the level of abstraction in the item formulation seems to play a role in polarizing respondents.

This finding can be interpreted in light of the construal level theory. According to the review by Trope and Liberman [37], “a high level of construal can be defined as a relatively abstract, coherent and superordinate mental representation of an object “. The extent to which an object has a high or low level of construal representation influences the intensity of affective response to the object [37]. Affective or stronger reactions to objects with a low construal level in answering survey items would manifest in selecting more extreme answer categories. Thus, they would result in opinion dissent or polarization at the aggregate level. An example from our items sample with a low construal level is “In your opinion, should the German government decide to supply battle tanks to Ukraine?”. An example from our items sample from the same issue domain with a high construal level is “Do you have a more positive opinion on the German military since the beginning of the war in Ukraine?”

Second, having psychological distance from an object reduces the tendency to respond to the object by drawing on one’s preconceptions while increasing openness to divergent viewpoints [38]. Manipulating objects to increase their level of construal representation has been shown to reduce opinion divergence between conservatives and liberals in the US: increasing the level of construal representation enhances “the capacity of both liberals and conservatives to consider each other’s positions and the potential to find common ground between them” [39]. In addition, high-level construal objects are more likely to require the activation of core values and ideals, such as fairness and justice, that are likely to be shared within important groups [40].

Activating such broader societal values, in turn, is expected to reduce partisan polarization insofar as everyone -regardless of partisan preferences- cares to some extent about fairness and justice [41]. In contrast, low-level construal objects include peripheral and contextual aspects, which enable context-dependent evaluations based on unique details of the present situation [40]. Attitudes towards low-level construal objects are therefore less likely to be convergent, as respondents build their opinion by retrieving context-dependent information and unique details of the object that are likely to be perceived and evaluated differently across respondents. Applying the construal level theory to items on policy-related issues implies that respondents overall agree on the goals of most policies (i.e., reducing social inequality or poverty) but disagree on the instruments (such as concrete policies) to reach these goals.

In sum, objects (in our case, survey items) with a high level of construal representation are less likely to attract affective or strong responses and to be based on preconceptions. At the same time, they are more likely to activate broader commonly shared values and openness to other viewpoints. This, in turn, can explain why high-level construal objects seem less polarizing than low-level construal objects. In other words, according to the construal level theory, survey items formulated in a more abstract way (high level of construal representation) are less likely to polarize than survey items formulated in a more concrete way (low level of construal representation). This leads us to formulate our fifth hypothesis:

1. H5: The more concrete the formulation of a survey item, the more it polarizes respondents.

Item level of technicality

Our last hypothesis focuses on the role of item level of technicality in distributive polarization. For this, we draw on both the work of Carmines and Stimson [42] on the distinction between easy and hard political issues and on the work of Converse [43] on voters’ level of political sophistication.

According to Carmines and Stimson [42], the distinction between easy and hard political issues is essential for a better understanding of issue voting. Easy issues imply so-called gut responses, “Because gut responses require no conceptual sophistication, they should be distributed reasonably evenly in the voting population” [42] (p. 49). Thus, all citizens–regardless of their level of political interest, political knowledge, or level of education–possess the ability to express their own opinion when answering such easy issue items. By contrast, the discriminatory power of hard issue items is likely to be higher among citizens who are more politically interested, informed, and involved than among citizens less interested, informed, and involved. In other words, it will be mostly citizens with a high level of political sophistication and political interest who will be able to give a valid answer expressing their own opinion on a hard issue. By contrast, citizens lacking political knowledge and involvement are more likely to give random answers to hard-issue items. Thus, item measurement error and consequently item variation are likely to be larger on hard issues than on easy ones. We would therefore expect more distributive polarization on easy issues than on hard issues as positions on easy issues are measured with more accuracy.

Converse [43] comes to a similar conclusion in his seminal work on the nature of belief systems in the mass public when analyzing the implications of varying levels of political sophistication among voters. According to him, respondents with a lower level of political sophistication show political positions that are more random and less structured than respondents with a higher level of sophistication. He argues that a lack of political information and contextual grasp among citizens leads to the inability to relate one’s ideology and own beliefs to a particular political issue. From an item perspective, items requiring a high amount of political and contextual information tend to show more variation and randomness in their answers than items not requiring such information [43].

Carmines and Stimson [42] conceptualized hard and easy issues by building on three complementary dimensions: level of technicality, measurement of policy ends and means, and length of the salience of an issue on the political agenda. We already drew hypotheses on two out of the three dimensions of easy and hard issues (i.e., hypothesis on level of abstraction -including the distinction between policy means and policy ends- and hypothesis on media issue salience). We, therefore, restrict this last hypothesis to the level of technicality of an issue, which enables us to construct a unidimensional indicator for measuring the easiness of an issue. Accordingly, technical issues require knowledge of important factual assumptions [42]. Hence, our last hypothesis is that the higher the level of technicality, the lower the level of polarization of an item:

1. H6: Items with a low level of technicality polarize more than items with a high level of technicality.

Sample

The German public opinion research institute Civey collected the data used for this study. Civey conducts online surveys through a network of partner websites, including prominent German online newspapers. The Civey panel comprises around one million German citizens who registered with their email addresses and created profiles with personal information such as age, gender, and zip code. Additional respondent details are collected over time. Civey utilizes nonprobability sampling and takes various measures to guard against self-selection biases, described in detail by Richter, Wolfram, and Weber [44]. All items underwent the standard Civey weighting procedure, combining quota sampling and poststratification across demographic and political variables. This procedure includes factors such as sex, age, voting behavior, regional density, and purchasing power of the zip region.

The original sample is composed of 1491 unique items that were asked to on average 5000 respondents between January 1, 2022, and October 14, 2022, date on which we retrieved the dataset. Ethical approval was not necessary as our analysis is based on the reuse of a dataset collected by the Civey research institute among its panel members who consented to participate. We had only access to meta information on each survey item (e.g., date of survey, item distribution, item formulation) without any access to respondents´ individual answer to the survey items. All items were asked with a five-point Likert scale. We restricted the sample to attitudinal items by using the following definition of attitudes: “attitude represents a summary evaluation of a psychological object captured in such attribute dimensions as good-bad, harmful-beneficial, pleasant-unpleasant, and likeable-dislikable” [45]. We, therefore, excluded items asking about knowledge or beliefs.

Furthermore, we restricted the sample to items measuring attitudes toward a political matter. We used Almond and Verba´s definition of the political system to distinguish political from non-political attitudinal items: “In treating the component parts of the political system we distinguish […] three broad classes of objects: (i) specific roles or structures, such as legislative bodies, executives, or bureaucracies; (ii) incumbents of roles, such as particular monarchs, legislators, and administrators, and (iii) particular public policies, decisions, or enforcements of decisions. These structures, incumbents, and decisions may in turn be classified broadly by whether they are involved either in the political or "input" process or in the administrative or "output" process” [46]. Furthermore, we excluded items beyond the context of Germany, the EU, Europe, or NATO. Lastly, we deleted all items asking about preferences toward one particular political actor without regard for the political actor´s behaviour or discourse (such as “How satisfied are you with Chancellor Olaf Scholz?”) as these items only measure party or personality preferences. Our restricted sample comprises 817 items that meet all mentioned selection criteria.

Operationalisation of the dependent variable

The agreement index proposed by Van der Eijk [47] constitutes a suitable measurement of distributive opinion polarization for ordinal variables, such as attitudinal items with Likert-scale answer categories. It has been widely used in previous studies on distributive opinion polarization [e.g., 11, 48]. The agreement index is computed with the proportion of respondents in contiguous answer categories. Applied to the example of a 5-point Likert scale, high agreement implies that most cases are found in contiguous answer categories, for example, the first and second ones. High disagreement, by contrast, means that observations fall into non-contiguous answer categories, for instance, 30% in the first answer category, 30% in the third answer category and 40% in the fifth answer category. S1 Appendix provides a more detailed description of its computation. The agreement index ranges from −1 to 1 and is presented in Graph 1. A value of −1 denotes complete polarization- (perfect bimodal distribution) where 50% of all respondents fall into the first and the other 50% into the last answer category. A value of 1 refers to complete consensus, where all observations are to be found in a single category (perfect unimodality). An index value of 0 refers to a uniform distribution, where each answer category comprises an equal number of responses (see Fig 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Van der Eijk index.

https://doi.org/10.1371/journal.pone.0302446.g001

Operationalisation of independent variables

We developed a comprehensive coding scheme to build the independent variables. Besides the continuous salience variables, all other independent variables are categorical with 2 to 3 categories. As the semiotic nuance between some sampled items is particularly granular, our coding scheme provides various examples for each variable category to ensure coding reproducibility. Furthermore, we computed an intercoder reliability test for each categorical variable based on a random sample of 100 items.

Intercoder reliability.

To assess the intercoder reliability, we ran a Cohen’s kappa test [50, 51]. Table 1 shows the kappa values for our independent variables.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Intercoder reliability.

https://doi.org/10.1371/journal.pone.0302446.t001

Following Landis and Koch’s [51] (p. 165) description of the relative strength of agreement associated with kappa (see Table 2), we obtained kappa values suggesting a substantial to almost perfect agreement.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Kappa and strength of agreement.

https://doi.org/10.1371/journal.pone.0302446.t002

Descriptive statistics

Fig 2 depicts the overall polarization of the 817 items we analyzed. The distribution is unimodal and somewhat skewed to the right, indicating a tendency to answer the questions related to the items in a consensual way. Fig 2 highlights an important result for the opinion polarization debate: only about 20% of the items have a negative value on the Van der Eijk index, and negative values on the Van der Eijk index are all moderate (with -0,55 being the minimum value on a scale theoretically ranging to -1). In other words, only about 20% of our 817 attitudinal items on political issues show moderate opinion polarization. At the same time, we do not find any attitudinal items with a high level of opinion polarization.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Distribution of all analyzed items on the Van der Eijk index.

https://doi.org/10.1371/journal.pone.0302446.g002

We gain a more precise insight by subdividing items’ value on the Van der Eijk index along their issue domains. Fig 3 illustrates the distribution of the Van der Eijk index along the items’ issue domain. The issue domains are ordered from least to most polarizing. Descriptively, we see that items related to gender issues, as well as to education, research, and infrastructure issues are the items with the most agreement (see top of Fig 3). By contrast, items on environmental, Europe-related, and health issues are the most polarized (see bottom of Fig 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Distribution of the items’ issue domains on the Van der Eijk index.

https://doi.org/10.1371/journal.pone.0302446.g003

To conclude the presentation of our dependent variable, Fig 4 brings some concrete insights into our sample of items with a list of items with the highest level of agreement, items with an exact uniform distribution, and items with the highest level of disagreement.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Top 5 items with the highest Van der Eijk value, top 5 items with a value closest to 0, and top 5 items with the lowest Van der Eijk value.

https://doi.org/10.1371/journal.pone.0302446.g004

Turning now to our independent variables, Table 3 presents their descriptive statistics. On average, the issue domain of an item is mentioned over 8.700 times in newspapers the week before the survey data collection of the corresponding item. Out of the 817 items, every tenth relates to a cultural issue and roughly every fifth has a notion of costs included in its formulation. Items outlining a benefit are less frequent (14%). In 17% of the cases, an item addresses a minority-related issue. While a third of the items shows a low level of abstraction, only every tenth item is formulated in a highly abstract way. Similarly, also only about ten percent of items shows a high level of technicality, while every fourth item implies only little knowledge to answer the item. Finally, roughly every tenth item was COVID-, or Ukraine-related.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Distribution of independent variables.

https://doi.org/10.1371/journal.pone.0302446.t003

Bivariate analysis of the Van der Eijk index

Fig 5 illustrates how the item characteristics impact distributive polarization by showing results of t-tests for nominal two-group comparisons (Hypothesis 2–4), variance analysis for multi-group comparisons (Hypotheses 5 and 6) and correlation for continuous variables (Hypothesis 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Effects of item characteristics on the Van der Eijk index.

Stars denote the significance level of pairwise comparison: *p<0.05; **p<0.01; ***p<0.001.

https://doi.org/10.1371/journal.pone.0302446.g005

First, salience is significantly and negatively associated with the Van der Eijk index, meaning that the higher the media salience of an item’s issue domain is, the more polarized an item is. This result is in line with our first hypothesis.

Second, an item belonging to the cultural issue domain positively affects the Van der Eijk index values (significant at the 5% level). In other words: if an item is culture-related, respondents are more likely to share consensual attitudes. This finding contradicts the postulated relationship we formulated in Hypothesis 2.

Third, items measuring attitudes toward financial and non-financial individual costs tend to have a significantly lower Van der Eijk index value than attitudinal items not assessing any costs. Furthermore, items measuring attitudes toward financial and non-financial individual benefits do not significantly differ in their Van der Eijk index value from attitudinal items not assessing any benefits. This finding is consistent with our Hypothesis 3: items measuring attitudes towards financial and non-financial individual costs polarize to a larger extent than items measuring attitudes toward financial and non-financial individual benefits.

By contrast, we don’t find evidence for our fourth hypothesis stating that items targeting a minority group are more likely to polarize: the association between the level of polarization and the extent to which an item tackles a minority-related issue is not significant at conventional levels. Fifth, highly abstract items polarize respondents significantly less than items formulated more concretely, corresponding to our Hypothesis 5. By contrast, there is no significant difference between items with a low and a medium level of abstraction. Lastly, items requiring a medium level of technicality tend to be more polarized than items requiring a low level of technicality. This is in contrast with our postulated relationship in Hypothesis 6. There is no significant difference in the level of polarization between items with a high level of technicality and those with a low or medium level.

As a robustness check and to better understand how item characteristics affect the level of polarization, we employ ordinary linear least squares regressions to regress the Van der Eijk index on our independent variables (S1 Table in S1 Appendix). Model 1 presents a regression model with all our explanatory independent variables, while we included the control variables capturing whether an item is COVID- or Ukraine-related in the second model. Overall, we can confirm our bivariate findings.