1. Introduction

Research conducted over the past decade has identified a number of new forms of influence that the internet has made possible. Some of these are among the largest effects ever discovered in the behavioral sciences, and they are of special concern because they can impact people without their awareness, because they often leave no paper trails for authorities to trace, and because they are largely controlled by unregulated monopolies [1–3]. Epstein and Robertson showed, for example, that search results that are biased to favor one candidate could shift the voting preferences of undecided voters by as much as 80% after just a single search experience on a Google-like search engine [1], and this effect has since been replicated partially or in full multiple times [4–11]. They also showed that this effect, called the “search engine manipulation effect” (SEME), can easily be masked so that users are unaware that they are viewing biased search results.

In the present paper, we describe and quantify yet another new form of online influence–the Targeted Messaging Effect (TME)–which has all of the most troubling characteristics of SEME and other recently identified forms of online influence [10–14]: it is a large effect; it can influence people without their awareness; it leaves no paper trail; and it is largely controlled worldwide by three unregulated monopolies–Facebook/Meta, Google, and Twitter.

Before we say more about TME per se, we will attempt to put our research on this topic into a larger context. Research on influence over human decision making has been conducted for over a century in multiple fields: business, psychology, sociology, political science, economics, and so on. In political science, for example, Paul F. Lazarsfeld’s classic studies in the 1940s and 1950s demonstrated the important role that “political predispositions” played in determining how people reacted to various forms of social influence, and, ultimately, in helping to determine how people voted [15–17]. Political scientists have also shown how voters are influenced by a wide range of factors, among them being the positive or negative connotation of a political message and the presence of a political candidate in media–newspaper coverage and television ads, for example [18–20]. Recent investigations show how voters are influenced by social media content, the online presence of a political candidate, and the perceived personability of a political candidate across different platforms [21–24].

Economists and business experts have developed numerous models to try to understand and predict consumer behavior [25,26]; once again, recent efforts have focused on how search engines, social media platforms, YouTube “influencers” and other new forms of influence made possible by the internet and other new technologies are impacting consumer choices [27,28]. Psychologists have been trying to understand decision making in broad terms applicable, perhaps, to all aspects of life, and they have been especially interested in recent decades in identifying extremely subtle forms of influence that are largely invisible to those affected [29–31].

We believe that SEME, TME, the Answer Bot Effect (ABE) [11], the Search Suggestion Effect (SSE) [12], and other new forms of influence that our research group has been studying over the past decade are fundamentally different than most forms of influence researchers have been studying over the years. Most forms of influence are inherently competitive: billboards, social media campaigns, television commercials, and print and online advertisements, for example. Even most of the shady forms of influence one sometimes reads about in headlines or novels are inherently competitive: ballot stuffing, the rigging of voting machines, vote buying, and so on [32,33]. Competitive forms of influence usually have little net effect for the simple reason that both (or all) sides can employ them. One manipulation might overpower the others when one side has more resources, but resources can shift over time.

The internet was envisioned by its founders to be a great leveler, giving every individual equal voice and giving small companies the ability to compete with giants [34,35], but it quickly evolved into an array of “walled gardens” [36,37] dominated by huge monopolies, each of which quickly gaining virtually exclusive control over specific forms of influence. Outside the Republic of China, Google (through its search engine and its property YouTube) controls access to most information, and Meta (through its properties Facebook, Instagram, and WhatsApp) guides the majority of online social interactions. TikTok has also become popular, accruing over 2 billion first-time downloads since its release in 2016, and it has even become a platform for “forming political coalitions” among young users [38–40]. Although far smaller than Google and Facebook, Twitter dominates the influential world of microblogging, especially in the United States [41,42].

The current walled-garden structure of the internet is highly problematic from an influence perspective. It means that if one of the large platforms favors one candidate, party, cause, or company, it can change people’s thinking and behavior on a massive scale without people’s awareness, without leaving a paper trail for authorities to trace, and without anyone having the means to counteract the manipulation. To be specific, if Google’s search algorithm boosted content in search results that favored Candidate A, unless systems were in place to capture such content–all of which is ephemeral–no one would ever know that this bias existed, even though, in a national election, it could conceivably shift the voting preferences of millions of undecided voters [1–3,10–14]. Even more disturbing, no one could counteract such bias. To put this another way, although two opposing campaign groups might battle each other to try to boost their visibility in search results or in YouTube sequences, no campaign organization has the means to counteract an action taken by or a policy implemented by the platform itself–by an executive, a rogue employee, an unattended algorithm, or some combination thereof. The problem worsens when these monopolies favor the same candidate or cause; patterns of campaign donations documented by organizations such as OpenSecrets.org in recent years suggest that major tech companies might in fact be politically aligned [43–45].

TME itself was presaged in a widely-read New Republic article by Harvard legal scholar Jonathan Zittrain [46]. As he noted, on Election Day in the US in 2010, Facebook sent go-vote reminders to 61 million Facebook users and, based on a nationwide analysis of voting records, subsequently concluded that its go-vote prompt had caused about 340,000 more people to vote than otherwise would have [47]. The prompt successfully nudged 0.57% of Facebook’s sample of eligible voters. That might not sound like much, but that proportion could easily swing a close election. Recall that Donald Trump won the Electoral College vote in 2016 because of a combined vote margin of only 79,646 votes in three US states [48]. If Mark Zuckerberg, CEO of Facebook, had elected to send vote reminders exclusively to supporters of Hillary Clinton on Election Day in 2016, that might have boosted the Clinton vote nationwide by more than 450,000; that number is based on a simple extrapolation from Facebook’s 2010 vote manipulation [49].

Zittrain’s concerns were legitimate, but, for four reasons, we believe that “digital gerrymandering” is an inappropriate label for this type of manipulation. First, gerrymandering–the relatively permanent redrawing of voting districts–and targeted messaging–the sending of consequential messages to only a subset of a larger group–have at best only one superficial characteristic in common: they each divide up a population in a way that serves the needs of an empowered group. But gerrymandering is a visible and relatively permanent manipulation–so visible and heavy handed that it is often challenged in court [50]. TMs sent to a subgroup online, however, are ephemeral. They impact people and then disappear. They are stored nowhere and cannot be reconstructed, which is why authorities cannot trace them. This is true of company-generated messages on Google’s home page, on YouTube (owned by Google), on Twitter, on Facebook and Instagram (owned by Meta), and other popular platforms. On YouTube, no records are kept of the sequences of videos shown to users, nor of that top video in the list, which is the “up-next” video that plays automatically unless the user selects a different video. On Twitter, company-generated tweets show only in the list you see when you first sign on; you can’t look at the tweets they showed you the last time you signed on.

And even though TMs can have a large impact on people’s opinions and votes (see below), virtually no one is aware that these messages are sent to some people and not others; without a large passive monitoring system in place that captures ephemeral content [51–53], no one can be certain that the manipulation even took place. Although some ephemeral political content was indeed being captured in the weeks leading up to the 2016 Presidential election [2,51], no one, to our knowledge, was tracking targeted messages sent by Facebook. Did Mr. Zuckerberg send out that go-vote reminder to Clinton supporters on Election Day? Unless he or a whistleblower comes forward to inform us, we will never know.

Other forms of online influence exist, of course, such as the influence exerted by thousands of bots launched by a secret organization in Russia to interfere with elections in the US [53–55], or micro-targeted ads posted by the now defunct company Cambridge Analytica in 2016 [56]. But manipulations like these–although occurring on our high-tech internet–are actually traditional in nature and are not, generally speaking, a threat to democracy. If Russian hackers launch a large number of anti-Biden bots, Biden’s party or another group of hackers could, in theory, launch its own bots to counter the Russian bots. This type of influence is very much like the influence exerted by billboards and television commercials: It is both visible and competitive [57], and as long as one has the resources, one can counteract it. Internet pioneers such as Tim Berners-Lee envisioned a future internet in which many thousands of relatively small entities would compete with each other for the attention of users [58], just as thousands of news media organizations have competed for people’s attention for a century or more. Unfortunately, as Berners-Lee himself has lamented in recent years, as the internet mushroomed in size, it became dominated (outside of mainland China) by “one search engine, one big social network, [and] one Twitter for microblogging” [59].

The dominance of such monopolies has put radically new and powerful means of influence into the hands of a small number of executives. For example, if Facebook–either through its main social media platform (S1 Fig) or through its subsidiary, Instagram (S2 Fig)–occasionally sends its users reminders to vote or reminders to register to vote, how would we know if these messages were being sent to all of its users or just to the members of one political party? The same could be said of Twitter, which currently inserts company-originated messages after every five or six tweets in people’s Twitter feeds, and of Google, which has been praised for including large “go-vote” messages on its home page on election days (S3-S5 Figs) [49,60]. If messages of this sort were being targeted to certain groups, unless a whistleblower came forward or a large-scale monitoring system was in place, we would not know, and, as we have noted, we would have no way to counter the manipulation.

Second, the term “digital gerrymander” already has a legitimate meaning in the social sciences. It refers to the use of computers to calculate optimal boundaries for voting districts [61,62]. Typically, this means boundaries that will virtually guarantee that one political party always wins. Computer modeling could also be used, of course, to guarantee maximum fairness in political redistricting, but that would rarely serve the interests of the people in power, and they are usually the people in charge of redistricting [63].

Third, the use of TMs for political purposes is just the tip of a very large iceberg. One immensely large class of TMs–targeted advertisements–impacts the purchases of millions of people every day. Nearly all of Facebook’s income comes from the fees companies pay to send their advertising content to targeted groups–people who appear, based on their Facebook profile and their most recent Facebook postings–to be highly likely to buy specific products from those companies. Because–at least in theory–any company can pay for that kind of advertising, it is inherently competitive and therefore no threat to consumers. But what if Facebook–in other words, the advertising platform–decided to ban certain ads or advertisers, or, more ominously, to throttle one company’s ads so that they often failed to reach the targeted audience? Again, without independent passive monitoring systems in place to capture the ephemeral content that actually reaches users, the manipulation of ads by platforms like Facebook and Amazon would be impossible to detect [2,51, cf. 64,65].

Fourth, targeted messaging–especially the messaging controlled by the large tech platforms themselves–can, in theory, influence almost any kind of thinking or behavior, not just political thinking or purchases. Targeted messages were certainly in wide use long before the internet was invented. Consequential messages have been delivered to specific groups of people on cigarette packs, condom boxes, pill containers–even on flimsy pieces of plastic used by cleaning companies to protect freshly cleaned clothes–and research has demonstrated the effectiveness of such messages, especially with certain populations [66,67]. The particular power that biased online messages have to alter thinking and behavior has also been demonstrated [68–70]. This is why we have set about trying to understand and quantify some aspects of the broader mechanism: Specifically, what happens when consequential messages are sent to one group and not another? How far apart can one push the groups? Will salient, high-contrast messages–that is, messages that stand out from a background–have a larger impact than subtle, low-contrast messages? Will the impact of a message increase if it is displayed multiple times? Can a single TM have a significant effect on people’s opinions and voting preferences? Do TMs on different platforms have comparable effects?

We will answer these questions in the experiments described herein. All four experiments employed procedures that were randomized, controlled, and double-blind, with all substantive content (such as the names and biographies of political candidates) counterbalanced to eliminate possible order effects.

2. Experiment 1: The impact of five low-contrast, verified, targeted messages on opinions and voting preferences

In our first experiment, we used a simulated Twitter feed to determine whether low-contrast, verified, targeted tweets could be used to shift the opinions and voting preferences of undecided voters. The appearance of these tweets closely matched that of the TMs the Twitter company currently sends to its users: (a) Our 5 TMs had a white background, just as our 30 organic tweets did. (b) The brief headline before the textual content read “Breaking News” in a black font. (c) The blue checkmark (signifying that Twitter had somehow “verified” the source of the tweet) was present on each TM, just as it is, at this writing, on all Twitter TMs (Fig 1). In order to assure that our participants would be “undecided,” we asked our US participants to express their views about two political candidates who ran for prime minister of Australia in 2019 [cf. 1].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. A screenshot showing an image of the first and second tweets in the Twitter feed employed in Experiment 1.

The first tweet was a targeted message coming presumably from the Twitter company itself, in this case containing positive information about Bill Shorten. It would thus have been shown to study participants in the Pro-Shorten bias group. Its format was low-contrast (white background, with a black “Breaking News” headline) and included a blue checkmark, signifying verification. The second tweet in the image was an organic tweet sent by a fictitious user.

https://doi.org/10.1371/journal.pone.0284495.g001

2.2 Results

Although participants were instructed to examine all the tweets in the Twitter feed (35 in the two bias groups, 30 in the control group), 29.0% of them did not comply, scrolling less than the full distance. Rather than discarding people with low scroll scores, we chose, for comparison purposes, to divide the sample into two groups: Low Compliance (maximum scroll values ≤ 50%) and High Compliance (maximum scroll values > 50%).

We call the shift in voting preferences “Vote Manipulation Power,” or VMP, which is the post-manipulation increase of people in the bias groups (expressed as a percentage increase) who said they would vote for the favored candidate [1]. For details about how VMP is calculated, see S4 Text. In the High Compliance group in Experiment 1, the VMP–the shift in voting preferences toward the favored candidate–was 87.0%, which is larger than any VMPs our team has ever found in SEME experiments [1]. A shift this large can, in theory, turn a 50/50 split among undecided voters into more than a 90/10 split (see S4 Text). The shift in the Low Compliance group–although smaller–was still substantial (Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Experiment 1: VMPs by compliance level.

https://doi.org/10.1371/journal.pone.0284495.t001

In the High Compliance group, answers to all six opinion questions shifted significantly in the direction predicted by the bias; in the Low Compliance group, answers to five of those six questions shifted significantly in that direction (Table 2), with the opinions shifting farther in the High Compliance group. Finally, the voting preferences as expressed on the 11-point opinion scale also shifted significantly and substantially in the predicted direction (see Table 3, where the data have been corrected for counterbalancing and candidate so that larger positive values indicate greater preference for the favored candidate).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Experiment 1: Pre- and post-manipulation opinion ratings of candidates.

https://doi.org/10.1371/journal.pone.0284495.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Experiment 1: Pre- and post-manipulation mean voting preferences on 11-point scale (corrected so that positive values indicate preference for the favored candidate).

https://doi.org/10.1371/journal.pone.0284495.t003

In the bias groups, only seven participants (out of 336, 2.1%) expressed concerns about possible bias in the content of the tweets; whereas 113 of these individuals (33.6%) commented specifically on the damaging (but never the positive) information in the biased TMs. Comments such as, “Read that Bill Shorten spent tax payer money, arrested for DUI and had an affair” and “Scott Morrison displayed a lot of bad judgment in his personal life (affairs, DUI arrests, etc.), which made me feel he was untrustworthy,” were common. People’s focus on the negative content in the TMs is addressed in Experiment 4 below, as well as in our Discussion section.

As we noted, Twitter’s TMs look almost exactly like the organic tweets of Twitter users. The main feature that consistently distinguishes the company’s TMs from most organic tweets is that their TMs all include the prestigious blue checkmark. In Experiment 2, we attempted to replicate our findings from Experiment 1 while omitting the blue checkmarks from our TMs.

3. Experiment 2: The impact of five low-contrast, non-verified targeted messages on opinions and voting preferences

3.1 Methods

3.1.1 Participants.

After cleaning, our participant sample for this experiment consisted of a new group of 532 eligible US voters recruited through the MTurk subject pool, screened once again by Cloud Research (see above). The cleaning procedure was identical to that of Experiment 1. Once again, the group was demographically diverse. See S1 Table for details about demographic characteristics. The mean familiarity level for our first candidate, Scott Morrison, was 1.10, and the mean familiarity level for our second candidate, Bill Shorten, was 1.04.

3.1.2 Procedure.

The procedure in Experiment 2 was identical to that of Experiment 1, with one exception: The blue checkmark was absent on our TMs (Fig 2). Since that feature consistently distinguishes Twitter’s TMs from most organic tweets, we sought to determine whether the absence of this feature might reduce the impact of TMs on opinions and voting preferences.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. A screenshot showing an image of the second and third tweets in the Twitter feed employed in Experiment 3.

The second tweet (top tweet in the image above) was a targeted message coming presumably from the Twitter company itself, in this case containing negative information about Scott Morrison. It would thus have been shown to study participants in the Pro-Shorten bias group. Its format was high-contrast (blue background, with a red “Tweeter Alert” headline).

https://doi.org/10.1371/journal.pone.0284495.g002

3.2 Results

As expected, although the vote shifts were still quite large in both the Low and High Compliance groups, VMP values dropped substantially when the blue checks were absent (Table 4) (VMP_Expt1High = 87.0, VMP_Expt2High = 61.7, z = 8.58, p < 0.001).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Experiment 2: VMPs by compliance level.

https://doi.org/10.1371/journal.pone.0284495.t004

Once again, opinions shifted in the predicted directions in all groups (Table 5), and so did mean voting preferences as expressed on the 11-point scale (Table 6). Even without the blue checkmarks on the TMs, participants also appeared to pay as much attention to them in Experiment 2 as in Experiment 1, with only 9 out of the 370 people in the bias groups (1.6%) raising concerns about possible bias in the content, and 115 of those people (31.1%) specifically mentioning the negative (but not the positive) things being said about the candidates in the TMs. The higher VMPs in Experiment 1 suggest that blue checkmarks add credibility to the content of the TMs, but the checkmarks do not seem to reduce the level of attention people are paying to them–or at least to the TMs with negative content.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Experiment 2: Pre- and post-manipulation opinion ratings of candidates.

https://doi.org/10.1371/journal.pone.0284495.t005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Experiment 2: Pre- and post-manipulation mean voting preferences on 11-point scale (corrected so that positive values indicate preference for the favored candidate).

https://doi.org/10.1371/journal.pone.0284495.t006

Could substantially boosting the salience of TMs in a Twitter feed increase their impact on people’s opinions and voting preferences? We explore this question in Experiment 3.

4. Experiment 3: Impact of high-contrast, verified targeted messages on opinions and voting preferences

4.2 Results

Because, generally speaking, increasing the salience of stimuli increases the attention they attract [73], one might expect that increasing the salience of the TMs would have increased their impact. The VMPs in Experiment 3, however, were significantly lower than the VMPs in Experiment 1, (Table 7) (VMP_Expt1High = 87.0, VMP_Expt3High = 81.1, z = 2.15, p < 0.05). Shifts in opinions (with one exception in the Low Compliance group) and voting preference as expressed on the 11-point scale also moved in the direction predicted by bias in the TMs (Tables 8 and 9), but, again, those shifts were lower than the ones we found in Experiment 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Experiment 3: VMPs by compliance level.

https://doi.org/10.1371/journal.pone.0284495.t007

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Experiment 3: Pre- and post-manipulation opinion ratings of candidates.

https://doi.org/10.1371/journal.pone.0284495.t008

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. Experiment 3: Pre- and post-manipulation mean voting preferences on 11-point scale (corrected so that positive values indicate preference for the favored candidate).

https://doi.org/10.1371/journal.pone.0284495.t009

Once again, comments focused largely on the negative TMs– 98 out of 313 people in the bias groups (31.3%) mentioned negative TMs, compared to only 1 person who mentioned positive TMs and only 6 people (1.9%) who commented on possible bias in the tweets.

The findings from Experiments 1, 2, and 3 suggest that Twitter displays its TMs the way it does–in a manner that makes them difficult to distinguish from organic user tweets–to maximize their impact on users.

This leaves us with (at least) two intriguing questions: To what extent can a single TM shift opinions and voting preferences, if at all? And how much more impactful might a single negative TM be than a single positive TM? We address these questions in Experiment 4.

5. Experiment 4: Impact of a single low-contrast, verified targeted message on opinions and voting preferences

5.2 Results

At first glance, the pattern of VMPs we found in Experiment 4 looks surprising (Table 10). In Experiments 1 to 3, the VMPs in the High Compliance groups were always substantially larger than the VMPs in the Low Compliance group. In Experiment 4 we found the opposite pattern, most likely because people in the Low Compliance group saw, on average, only 36.2% of the tweets following the TM in position 2, whereas people in the High Compliance group saw, on average, 97.4% of the tweets following that TM. Exposure to a large number of relatively bland tweets following a biased TM apparently dilutes the power of that TM. Finally, once again, very few people claimed that they saw any bias in the Twitter feed we showed them; only 3 out of the 399 people in the bias groups (0.75%) expressed concerns about possible bias in the content, and 76 of those people (19.0%) specifically mentioned the negative (but not the positive) things being said about the candidates in the TMs.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 10. Experiment 4: VMPs by compliance level.

https://doi.org/10.1371/journal.pone.0284495.t010

Breaking down the impact of positive TMs versus negative TMs on the VMPs in Experiment 4 confirms the enormous power that negative content has to alter people’s thinking (Table 11). The positive TMs had virtually no impact on VMPs in either the Low Compliance or High Compliance groups. The negative TMs, on the other hand, had a relatively large impact on High Compliance participants (VMP = 51.2%) and shifted all of the 17 Low Compliance participants (VMP = 100.0%). Only one of the 399 people in the bias groups expressed any concerns about possible bias in the tweets (0.003%), whereas 76 of these individuals (19.0%) specifically singled out the negative content of the single TM (regarding the candidate’s DUI conviction) as a reason for not supporting him. No participants mentioned the contents of the positive version of the TM (regarding the candidate receiving The Innovation in Politics Award) in their typed comments. The possibility of bias was mentioned somewhat more frequently in comments in Experiments 1 through 3, presumably because people in the bias groups in those experiments saw as many as five TMs that shared the same bias; in Experiment 4, people saw only one TM.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 11. Experiment 4: VMPs by type of TM (positive or negative).

https://doi.org/10.1371/journal.pone.0284495.t011

Most opinion shifts in the bias groups in Experiment 4 occurred in the predicted direction (Table 12), but they were smaller than the shifts found in the earlier experiments, presumably because participants had less information on which to base their opinions. Changes in voting preferences as expressed on the 11-point scale also occurred in the predicted direction, but, again, they were smaller than in the previous experiments (Tables 13 and 14).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 12. Experiment 4: Pre- and post-manipulation opinion ratings of candidates.

https://doi.org/10.1371/journal.pone.0284495.t012

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 13. Experiment 4: Negative TM Pre- and post-manipulation mean voting preferences on 11-point scale (corrected so that positive values indicate preference for the favored candidate).

https://doi.org/10.1371/journal.pone.0284495.t013

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 14. Experiment 4: Positive TM pre- and post-manipulation mean voting preferences on 11-point scale (corrected so that positive values indicate preference for the favored candidate).

https://doi.org/10.1371/journal.pone.0284495.t014

Experiment 4 suggests that a single biased TM in a Twitter feed can impact people’s decision making, at least as it pertains to political candidates running for office.

6. Discussion

Recent news about the Twitter company is relevant to our research findings. According to an August 23rd, 2022, investigative story in the Washington Post [74], “an explosive whistleblower complaint” from Peter Zatko, former head of security at Twitter–an 84-page document filed simultaneously with the Securities and Exchange Commission, the Federal Trade Commission, and the Department of Justice [75]–Twitter had lax security that allowed false content to be posted easily by hackers, bots, foreign powers, and company employees. Regarding employees, the Post reported that “about half of Twitter’s roughly 7,000 full-time employees had wide access to the company’s internal software and that access was not closely monitored, giving them the ability to tap into sensitive data and alter how the service worked.” According to Zatko, Twitter algorithms also determined what content gets suppressed or “amplified” [76].

Given Elon Musk’s purchase of the company in October, 2022 [77] and his subsequent firing of most of Twitter’s employees, Zatko’s concerns about the security of the company’s operations might understate the nature of the problems that might be emerging in a new and relatively unstable version of the company. Given the apparent power that tweets–especially tweets containing negative content–can have on opinions and voting preferences–we believe that Twitter’s operations should be examined closely not only by Twitter’s corporate leaders, but also by government officials and public policy makers in countries worldwide. Twitter currently has 480 million daily users, and it serves as an official platform for world leaders, government agencies, news services, and thousands of companies and organizations; even Pope Francis has a Twitter account. All those Twitter feeds are vulnerable to hacking and hijacking, according to Zatko’s complaint, which contains examples of such interference.

Our experiments suggest that TME is a remarkably large effect, especially when Twitter itself sends people sensational tweets that have certain visual properties (Experiment 1): tweets with white backgrounds (matching the backgrounds of organic tweets), a brief headline (such as “Breaking News”), and Twitter’s trademark blue checkmark. Experiment 1 yielded a VMP of 87%, with only 2.1% of the participants in the two bias groups expressing any concerns about possible bias in the Twitter feed we showed them. That VMP shift means that in a group of 1,000 undecided voters–split, by definition, 500/500 before exposure to a biased Twitter feed–after viewing that feed, the split will now be 65/935, which means that interacting with the Twitter feed changed a win margin of 0% to a win margin of 87% among vulnerable voters. That shift could occur, in theory, with nearly 98% of the people in such a group having no idea they were manipulated.

On its face, a shift that big might seem impossible. In the real world, certainly, people are being influenced by many sources of information, not just by Twitter, and we currently have no reason to believe that Twitter’s content is systematically biased to support just one candidate or political party. But our experiments show the potential that Twitter feeds have to shift opinions and votes. Twitter is a private company that is not accountable to the public, and no laws or regulations exist at this writing that would in any way restrict Twitter’s ability to send highly biased content to users. Indeed, some people have claimed that Twitter’s content already shows significant political bias at times [cf. 76]. Trump supporters cried foul, for example, when Twitter permanently shut down the President’s Twitter account just after the January 6, 2021 insurrection in Washington, D.C. [78], and objections were raised when Twitter apparently suppressed news stories related to content found on Hunter Biden’s laptop computer in October 2020 [79]. In this case, some of the facts about the laptop originally reported by the New York Post on October 14, 2020 were subsequently confirmed by both the New York Times and the Washington Post [80, cf. 81]. That Twitter content might show political bias should surprise no one given that, according to OpenSecrets.org, more than 96% of donations from Twitter and its employees in recent years have gone to one political party [43].

No experiments can show that a source of influence like TME is actually being used. Since 2016, however, our team has been building increasingly larger and more sophisticated systems that capture the ephemeral content being shown to users by Google, YouTube, Bing, Yahoo, and other companies [51,52,82]. In 2020, we preserved and analyzed more than 1.5 million online ephemeral experiences that would normally have been lost [52]. In 2022, we preserved more than 2.4 million online ephemeral experiences related to the US midterm elections, including, this time around, content from Twitter which we are currently analyzing.

We acknowledge that if, at some point, we detect political or other bias in Twitter feeds being displayed to certain groups, that will still tell us nothing about the origin of such bias. Bias in ephemeral content can be programmed deliberately [83], generated by unconscious bias on the part of programmers [84], or generated by user behavior [85]. No matter what the original of such bias, given the apparent power it has to shift opinions and voting preferences, we believe that if large-scale bias is ultimately found to exist in actual Twitter feeds, this is an issue that Twitter executives and government officials will need to examine. Otherwise, extreme bias–especially bias targeted toward certain groups–could easily undermine the integrity of the free-and-fair election. Moreover, if monitoring systems are not in place to preserve ephemeral content such as Twitter feeds, democracy might be undermined without the electorate knowing. Based on Mr. Zatko’s recent revelations [74,75], along with documented cases in which Twitter content has been hacked by bad actors [86,87], it now appears that extreme bias in Twitter content can be introduced fairly easily by agents of foreign powers, by aggressive Twitter employees, or even by mischievous teenagers [74,75].

Supporting information

Acknowledgments

We thank M. Voillot for help in preparing and running the experiments and V. Zankich and C. Sutlick for help in preparing the manuscript.