Bots, disinformation, and the first impeachment of U.S. President Donald Trump

Automated social media accounts, known as bots, have been shown to spread disinformation and manipulate online discussions. We study the behavior of retweet bots on Twitter during the first impeachment of U.S. President Donald Trump. We collect over 67.7 million impeachment related tweets from 3.6 million users, along with their 53.6 million edge follower network. We find although bots represent 1% of all users, they generate over 31% of all impeachment related tweets. We also find bots share more disinformation, but use less toxic language than other users. Among supporters of the Qanon conspiracy theory, a popular disinformation campaign, bots have a prevalence near 10%. The follower network of Qanon supporters exhibits a hierarchical structure, with bots acting as central hubs surrounded by isolated humans. We quantify bot impact using the generalized harmonic influence centrality measure. We find there are a greater number of pro-Trump bots, but on a per bot basis, anti-Trump and pro-Trump bots have similar impact, while Qanon bots have less impact. This lower impact is due to the homophily of the Qanon follower network, suggesting this disinformation is spread mostly within online echo-chambers.


Introduction
On December 18, 2019, the United States House of Representatives voted to approve articles of impeachment against President Donald Trump.The resulting trial in the Senate concluded on February 5, 2020 when the Senate voted to acquit the president.
During this period, online social media platforms became a battlefield for information warfare between supporters and opponents of the president [1].
While much of the activity originated from users engaging in genuine political debate, a significant proportion came from accounts known as bots.Social bots are automated social media accounts programmed to share certain content and interact with other users [2].Different kinds of bots are programmed for different purposes, including traditional spam bots which aggregate news content or distribute links [2], financial bots which advertise commercial products or attempt to influence financial markets [2], "astroturf" bots which promote political figures and their policies [2,3], and March 28, 2023 2/37 fake follower accounts used for inflating a user's follower network [2].
Due to their automation, social bots have the potential to manipulate social media discussions [2,4,5].The use of bots to spread targeted political messages online, known as computational propaganda [6], has been growing in recent years.Studies have chronicled efforts by bots to manipulate online discussions surrounding U.S. elections and political events since 2016 [7][8][9][10][11][12][13].
Recent trends show bots amplifying and spreading false or misleading news stories and conspiracy theories (collectively known as disinformation).The phenomenon of bots spreading disinformation has been observed in online discussions ranging from U.S.
elections [13] to public health issues such as vaccines [14,15] and the COVID-19 pandemic [16].The introduction of disinformation makes the risk posed by bots much greater, as it allows bots to create false narratives that take hold with a large population, resulting in dangerous outcomes.For example: on December 4, 2016, an armed believer of the "Pizzagate" conspiracy theory fired his weapon in a D.C. pizza parlor [17][18][19][20]; and on January 6, 2021 a mob of Trump supporters influenced by the "Qanon" conspiracy theory would storm the U.S. Capitol building in an effort to forcibly overturn the results of the 2020 U.S. presidential election [21][22][23].Due to the unique threat bots pose, it is important to be able to identify the bots and quantify their impact in spreading disinformation.
There is a large body of research on bot detection.Many bot detection methods rely on analyzing characteristics and behaviors of individual accounts.Some methods in this category focus on the temporal behavior of accounts [24,25].Others focus on the text posted by the accounts [26][27][28].Many approaches combine a variety of features of an account and use them as input to a machine learning classifier [29][30][31][32].The most prominent method in this class is the Botometer (formerly BotOrNot) [33] which uses a machine learning model applied to all data of an account (tweets, profile, followers).
It has been observed that methods which look at accounts individually may not be able to detect evidence of coordination between groups of accounts [31,36,37].For this reason, other methods have been developed which focus on analyzing collective behaviors and similarities among groups of accounts [36][37][38][39][40], or including such analysis March 28, 2023 3/37 in a hybrid "tiered" approach [41].It has been found that some of these collective behavior based methods outperform methods which look at accounts individually [40].
In terms of attempts to quantify bot impact, existing studies provide simple statistics such as the number of bots or volume of content they share [11,12].However, these statistics do not incorporate the interaction of social network structure, user activity levels, and user sentiment.Some studies have looked individually at the positioning of bots within the social network [42] or the bot retweet response time [4], but not the interaction of these factors.Recent work has presented a novel network centrality measure known as generalized harmonic influence centrality that combines all of these factors to assess bot impact in online political discussions [40,43].This centrality measure provides a better measure of bot impact than individually examining each factor.
In this study, we focus on social bots discussing the first impeachment of U.S.
President Donald Trump on Twitter.We find that bots are 66 times more active than normal human users, producing nearly one third of all impeachment-related content, despite representing less than 1% of all accounts discussing the impeachment.Bots tend to share news from lower quality sources (including disinformation) than their co-partisan human counterparts.Bots have an unusually high prevalence among Qanon conspiracy supporters, indicating that efforts are being made to artificially amplify this disinformation.Using generalized harmonic influence centrality, we show that pro-Trump and anti-Trump bots have a similar level of impact per bot, but Qanon bots have a lower per bot impact.Analysis of the Qanon follower network suggests this lower impact is due to Qanon users existing in an online echo-chamber with a high amount of ideological homogeneity.

Results
Our analysis contains multiple steps, which are shown in Fig 1 .We begin by collecting social media data related to the impeachment.Then we apply a variety of methods to classify the accounts by partisanship, bot status, and Qanon support.We perform a variety of analyses comparing the different types of accounts in terms of their posted content and network structure.Finally, we quantify the impact of the bots using

Content Analysis
Bot Impact Analysis

Account classification
We classified the Twitter accounts into different groups which characterized different aspects of their preferences and behavior.These included political partisanship, support for the Qanon movement, and whether or not the account was an automated bot.Each group label was assigned using a different method.A summary of the group statistics is provided in Table 1.First, we considered the political partisanship of the accounts.We trained a bidirectional encoder representations from transformers (BERT) model to measure the political partisanship of any given tweet text [44].The model was trained using a subset of the impeachment tweets for which we were able to assign ground-truth labels (see Methods).A label of zero represents strong anti-Trump sentiment and a label of one represents strong pro-Trump sentiment.We used the trained model to measure the sentiment of all tweets in our dataset.Then, we assigned a partisanship score to each Twitter account equal to the mean value of the partisanship score of their tweets.The accounts were labeled anti-(pro-) Trump if their partisanship score was less than or equal to (greater than) 0.5.In total, our dataset had 2.3 million anti-Trump accounts and 1.3 million pro-Trump accounts.This left leaning bias in the Twitter conversation aligns with findings from previous studies [40].
We next identified accounts who were supporters of the Qanon movement, based on those who promoted Qanon hashtags used in their profile descriptions (see Methods).
Using this process, we identified 25,360 Qanon accounts.
only their retweet network (see Methods).We chose this algorithm because it had minimal data requirements compared to other algorithms [33] (as we are easily able to construct the retweet network from the collected tweets), and also because it exhibited performance similar to or better than other algorithms [40].We applied the bot detection algorithm to daily retweet networks constructed from tweets posted each day.This allowed us to identify daily sets of active bots.The algorithm we use is designed to identify bots that engage in excessive retweeting.These retweet bots produce a disproportionate volume of social media content that can distort online discussions, so we focus our analysis on this type of bot.Our detection algorithm will not identify bots which exhibit different behavioral patterns that do not consist of excessive retweeting.
Therefore, when we classify an account as a bot, we specifically mean a bot retweeting at an unusually high rate.We note that one limitation of our bot detection algorithm is that it relies upon the retweet network, and so can only detect bots who retweet someone.However, since our focus is on retweet bots that amplify certain voices, this limitation is not a major issue.
Our final set of bots consisted of the union of these daily bot sets.In total we found 24,150 bots, of which 10,145 were anti-Trump and 14,005 were pro-Trump.This ratio of bots to humans aligns with other studies of U.S. politics on Twitter [13,40].
We summarize the prevalence of bots in different groups in Fig 2. We see that bots have a slightly higher prevalence among pro-Trump accounts than anti-Trump accounts (p-value < 10 −6 ).However, the bot prevalence among Qanon supporters is nearly an order of magnitude larger than it is among normal accounts (p-value < 10 −6 ).This suggests malicious actors may be attempting to use artificial accounts to amplify Qanon content.On one hand, this may be reassuring, as it suggests there aren't as many real Qanon supporters on social media as there may appear.On the other hand, significant bot presence means that Qanon content is being spread at a higher rate and with a potentially higher reach than could be achieved with humans alone.This high bot fraction is even more concerning when looking at the tweet rate of the accounts in

Content analysis
There are differences in the nature of content posted by the various account types.We first considered the quality of news stories shared by the accounts.We leveraged a previously published set of 60 news sites (20 mainstream, 20 hyper-partisan, and 20 fake news, with liberal and conservative leaning sites in each category) whose trustworthiness had been rated by eight professional fact-checkers [45].We followed the approach used in prior work [46,47] and calculated a media quality score for each user by averaging the trustworthiness ratings of any of their impeachment related tweets that contained links to any of those 60 sites.There was a link of some sort in 11% of the tweets, and of those with links, 12% had a link from one of the 60 news sites.In total, we were able to calculate a media quality score for 217,692 users.The media quality score ranges from one to five, with higher values indicating higher trustworthiness.Like other researchers in this space [46][47][48][49], we use source trustworthiness as a proxy for article accuracy, because it is not as feasible to rate the accuracy of every shared link.We note that our analysis of news quality is based on accounts in our dataset who share news stories, so care must be taken when generalizing to a broader set of accounts.Our conclusions March 28, 2023 8/37 apply only to the subset of accounts of each type who choose to share news stories.striking observation is that the media quality score is much higher for anti-Trump accounts than pro-Trump accounts.This is consistent with past work which found that pro-Trump (Republican) users were much more likely to share news from untrustworthy news sites than anti-Trump (Democratic) users [49].We also find that within each partisan group, the media quality score of bots is lower than that of humans (p-value In addition to what news media the accounts are sharing, we also investigate the tone and sentiment of the tweets they post.One measure of this is known as toxicity, which captures how harmful or unpleasant a tweet is.We measured the toxicity of all tweets in our dataset using Detoxify [50], a neural network model for toxicity detection that has achieved similar performance to top scorers in multiple Kaggle competitions related to detecting toxic language in web-based content [51].Detoxify has been used in several studies for profiling hate speech spreaders [52], detecting cyberbullying in online forums [53], establishing toxicity thresholds for content [54], manipulating language March 28, 2023 9/37 models [55], video-text retrieval systems [56], and measuring the safety of conversational neural network models [57].
Despite its high performance, the model is sensitive to insults and profanity, which may result in higher toxicity scores regardless of the intent or tone of the message.The model produces toxicity scores ranging from zero to one, with higher values indicating higher toxicity.We applied Detoxify directly to the tweet text.If the tweet is a retweet, then Detoxify is applied to the text of the retweet.We do not open any links in a tweet or measure the toxicity of the destination website.We feel this is sufficient as a link does not have the same impact on an online conversation as toxic text in a tweet.
The average toxicity score for each account type is shown in When we focus on the bot accounts, we find two general patterns.Within each partisan group, bots share lower quality media than humans and post less toxic content than humans.From this we can deduce that bots mainly share low quality news stories (relative to their human counterparts within their partisan group), but they tend not to amplify negative messages which use aggressive language.This suggests that the bots are more focused on spreading information and not on agitating users with toxic posts.Retweeted status text Bot count RT @SpeakerPelosi: The House cannot choose our impeachment managers until we know what sort of trial the Senate will conduct.President Trump blocked his own witnesses and documents from the House, and from the American people, on phony complaints about the House process.What is his excuse now?

4,983
RT @RepAdamSchiff: Lt. Col. Vindman did his job.As a soldier in Iraq, he received a Purple Heart.Then he displayed another rare form of bravery -moral courage.He complied with a subpoena and told the truth.He upheld his oath when others would not.Right matters to him.And to us.

4,225
RT @RepAdamSchiff: Impeachment of a president is a serious undertaking.The Senate's role is to act as an impartial jury and provide a fair trial.Fair to the President and to the American people.That means seeing all the evidence, documents and witnesses.What is McConnell afraid of?

4,212
RT @SpeakerPelosi: In the Clinton impeachment process, 66 witnesses were allowed to testify including 3 in the Senate trial, and 90,000 pages of documents were turned over.Trump was too afraid to let any of his top aides testify & covered up every single document.The Senate must #EndTheCoverUp

3,977
RT @SpeakerPelosi: The President & Sen. McConnell have run out of excuses.They must allow key witnesses to testify, and produce the documents Trump has blocked, so Americans can see the facts for themselves.The Senate cannot be complicit in the President's cover-up.#DefendOurDemocracy

3,955
The messages retweeted by the greatest number of anti-Trump bots, and the number of anti-Trump bots who retweeted each.Retweeted status text Bot count RT @realDonaldTrump: I was very surprised & disappointed that Senator Joe Manchin of West Virginia voted against me on the Democrat's totally partisan Impeachment Hoax.No President has done more for the great people of West Virginia than me (Pensions), and that will....

8,265
RT @realDonaldTrump: "Nancy Pelosi said, it's not a question of proof, it's a question of allegations!Oh really?" @Judge-Jeanine @FoxNews What a disgrace this Impeachment Scam is for our great Country!

8,134
RT @realDonaldTrump: As hard as I work, & as successful as our Country has become with our Economy, our Military & everything else, it is ashame that the Democrats make us spend so much time & money on this ridiculous Impeachment Lite Hoax.I should be able to devote all of my time to the REAL USA!

8,059
RT @realDonaldTrump: Crazy Nancy Pelosi should spend more time in her decaying city and less time on the Impeachment Hoax!

7,645
RT @realDonaldTrump: Many believe that by the Senate giving credence to a trial based on the no evidence, no crime, read the transcripts, "no pressure" Impeachment Hoax, rather than an outright dismissal, it gives the partisan Democrat Witch Hunt credibility that it otherwise does not have.I agree!

7,473
The messages retweeted by the greatest number of pro-Trump bots, and the number of pro-Trump bots who retweeted each.

Network analysis
Bots and Qanon supporters have networks that exhibit distinct properties.We first consider the bot follower network.The Qanon follower network, comprised of Qanon bots and humans, has an interesting structure.We find that this network contains a core of bots which are connected to each other, surrounded by a periphery of humans.Interestingly, these humans are only connected to the bots, and not to each other.We show this network in Fig 6, where the nodes are laid out with bots in the center to highlight this structural property.The connectivity among bots and lack of connectivity among humans suggests there is a hierarchical structure within the Qanon community.The bots act as sources of content for the humans.The humans appear not to engage with each other, but rather mainly consume content from the bots.We will see later that this network structure has implications for the impact of Qanon bots.

Bot impact analysis
Recall that our focus is on bots that retweet excessively.We next provide a quantitative measure of the impact of these retweet bots on the impeachment discussion.Thus far we have presented a detailed analysis of the activity, sentiment, and network structure of the bots.Impact is a combination of all of these factors.We would expect more active bots to have more impact, as they post more content.Bots with larger network reach should also have greater impact.Finally, bots whose followers are not strong co-partisans would have more impact because they can persuade these followers to their side.In contrast, strong co-partisan followers who already agree with the bots likely cannot be persuaded further.A measure that combines these factors is known as harmonic influence centrality [58].This is a network centrality based on a classic model for opinion dynamics [59] which incorporates stubborn users with immutable opinions [60].Harmonic influence centrality measures how much a set of nodes shifts the average equilibrium opinion in the network.The centrality naturally incorporates activity, network reach, and opinions to measure the impact of nodes in a network because it is based on an opinion dynamics model which utilizes these factors.In its original form, harmonic influence centrality used only the network structure and activity level.A more recent version, known as generalized harmonic influence centrality (GHIC), incorporates additional data, such as the node sentiment, making it more appropriate for real social networks.GHIC has been used to quantify the impact of bots March 28, 2023 15/37 in Twitter networks discussing various geo-political events [40].Because our data is quite similar in nature, we use this generalized version of the measure to quantify the impact of the bots.
To apply GHIC, one first must define the network.We want to calculate a daily impact measure for the bots to see how their impact evolves over time.Therefore, the networks we use are daily active follower networks.For a given day, the daily active follower network is the sub-network of the entire Twitter follower network induced by accounts which are active (post at least one tweet) that day.We follow the approach in [40] to calculate the GHIC of different groups of bots on these networks (see Methods).
We first look at the GHIC of all bots.The daily GHIC of all bots is shown in Fig 7, where positive values indicate a shift toward pro-Trump opinions, and negative values indicate a shift toward anti-Trump opinions.This shows on a given day which partisan side of bots had greater impact.We observe that for most days, the anti-Trump bots have greater impact.However, there are days when the pro-Trump bots have greater impact.Upon closer investigation, we find that on these days there was a news story which excited pro-Trump users on Twitter.For instance, on December 29, 2019 Donald Trump posted a controversial tweet referring to U.S. Speaker of the House of Representatives Nancy Pelosi as "Crazy Nancy".We see that the GHIC had a large positive value that day, indicating that the pro-Trump bots had greater impact.In general, we find that the GHIC achieves a large magnitude on days with polarizing events.
In addition to the daily GHIC, we are also interested in the efficiency of each group of bots.By efficiency, we mean the GHIC per bot for a group of bots.This measure would identify which group of bots are more impactful, on average.We consider three non-Qanon bots (p-value < 10 −6 ).To understand why Qanon bots are less efficient, we look at Fig 6 .We saw there that Qanon bots have a higher fraction of co-partisan followers than non-Qanon bots, meaning that Qanon bots are more ideologically aligned with their followers.Because of this, these bots cannot impact their followers' opinions as much, which lowers their GHIC efficiency.Our finding suggests that Qanon bots are not as effective at persuasion.Rather, they likely preach to a converted audience, which diminishes their impact compared to non-Qanon bots.
March 28, 2023 tweets, posted by 3.6 million unique Twitter users.Keywords used to collect impeachment related tweets, and the date each was added to the collection.
As a part of this tweet collection process, we also collected the Twitter profile of each user.The profile included information such as the name of the user, location (if provided), and a short description provided by the user.
We later used the Twitter Search API to collect user follower networks in a separate collection process.Our network convention was to have follower edges point from a user to the person that followed them.This way the follower edges point in the direction of information flow, as tweets from a user appear in the timeline of their followers.To build the follower network for the users in our dataset, we used a customized web crawler to collect a list of followings for each user (i.e. the users they follow).We chose to collect the followings rather than the followers for each user because it reduced our data collection burden.We observed that the follower count can be much larger than March 28, 2023 On the held out testing data it achieved a 96.3% accuracy score.We show the confusion matrix on the test data in Fig 8 .As can be seen, the classifier achieves high true positive and negative rates while maintaining low false positive and negative rates.Also, the rates are nearly equal for both classes.We provide some random samples of tweets from each end of the sentiment spectrum and their partisan sentiment measured by the classifier in Table 7.We plot in

Qanon classification
To identify Qanon supporters, we utilized a list of terms commonly used by Qanon supporters at the time, which are shown in Table 8.The terms on this list are consistent with those discussed by other work in this field [23,64].Other researchers have classified Qanon supporters based on hashtags used in their tweets or retweets [23],   that included at least one of these terms in their profile description was labeled as Qanon (after excluding any anti-Trump accounts or accounts which tweeted using anti-Trump hashtags).A limitation of this approach is that our list of hashtags is likely not comprehensive of all Qanon hashtags, especially as they may evolve over time.Another popular algorithm is the machine learning based Botometer (formerly BotOrNot) [33] which utilizes a large amount of data about an individual account, including followers, friends, tweets, and profile, in order to determine if it is a bot.
Botometer is a good algorithm to use when one wants to determine if an individual account is a bot.However, when one wants to find bots among a large set of users discussing a topic, as is the case in our impeachment analysis, the factor graph algorithm is more convenient.We can identify bots with this algorithm without having to collect any additional data, which is important given how large our daily active user sets are.In addition, it has been found that the factor graph algorithm has slightly better performance than Botometer [40].
The factor graph algorithm first constructs a retweet network based on the retweets in a given set of tweets.In this retweet network the nodes are the users who tweet and an edge (u, v) pointing from user u to user v means that v retweets u.The edge (u, v) is given a weight w uv which equals the number of times v retweets u.Like with the follower network, the edge direction in the retweet network indicates the flow of information.The algorithm uses the retweet network structure to calculate a bot probability for each user.This probability is based on the empirical observations that bots are likely to retweet humans, but unlikely to retweet bots, and humans are likely to retweet humans and less likely to retweet bots [40].This homophily for the humans and heterophily for the bots is utilized by the algorithm to determine bot probabilities from the structure of the retweet network.
We used the factor graph algorithm to identify bots active each day.To apply the algorithm, we first constructed a retweet network from the tweets posted on a given day.
Once the retweet network is constructed, we apply the algorithm to the network using the parameters specified in [40] to simultaneously obtain the joint bot probability for all users.We show one example of the bot probability distribution for a single day in Fig 10 .As can be seen, the bulk of users have bot probabilities near 0.5, which is the algorithm saying it cannot determine one way or another what the user is.In the upper probability range we see that the histogram decreases up to approximately 0.8, and then increases afterwards.This suggests there is a cluster of nodes with bot probabilities in the interval [0.8, 1.0].We used the lower probability bound of this cluster as the bot probability threshold so that this cluster of nodes is identified as bots.
March 28, 2023 25/37 We found a similar behavior in the bot probability distribution across all days.
Therefore, we used the same bot probability threshold of 0.8 for each day.Any user who had a bot probability over the threshold in at least one day in our dataset was declared to be a bot.We use this approach because due to automation, it is easy for a bot to act like a human (retweet less frequently).However, it is harder for a human to behave like a bot (retweet at an extremely high rate).Also, bots may intentionally behave in this non-constant manner.A bot may be extremely active on certain days when it is amplifying certain users, and quiet or only slightly active on others.Humans showing elevated retweet rates may have difficulty being as active as a bot.Under these assumptions, exhibiting bot behavior on at least one day would indicate the account is a bot, and humans would not show bot-like behavior any day.Across all days we found 24,150 bots among 3.6 million active accounts.As a terminology note, in this paper when we refer to the word "bot" we mean the accounts likely to be retweet bots, and when we refer to the word "human" we mean accounts not likely to be retweet bots (i.e. March 28, 2023 26/37 non-bots).
To explore the validity of our bot classifications, we took a random sample of around 7500 accounts with a roughly 60/40 human to bot ratio, and compared them against bot scores obtained from the Botometer API [3].The Botometer API provides scores from 0 to 1 representing the likelihood that a given account is a bot (i.e."overall" and "cap" scores), as well as separate scores for a number of bot sub-types (e.g."astroturf", "financial", etc.).We construct receiver operating characteristic (ROC) curves for different Botometer scores and our labels as a ground-truth.We use area under the curve (AUC) score to measure the agreement between our bot classifications and each kind of score provided by Botometer.We find that the AUC score for the "overall" bot scores is around 0.79, while the AUC score for the "astroturf" scores is much higher, around 0.93 (see Fig 11).This suggests that our bot detection algorithm, when used within the context of a political discussion, performs generally well at identifying bots overall, and performs even better at identifying hyper active political bots.From this figure we see that there are clusters of humans with very high toxicity scores.

Distribution of toxicity scores
The bots do not have such a toxic cluster.To further quantify these clusters, let us define a toxic user as one whose average toxicity score is greater than or equal to 0.9.
March 28, 2023 27/37 With this definition, we find that 2% of Republican humans are toxic, while 6% of Democrat humans are toxic.There are zero toxic bots under this definition.

Human Bot
Account Type

Generalized harmonic influence centrality calculation
To calculate the generalized harmonic influence centrality (GHIC) of a set of nodes we need several pieces of information.First, we need the follower network of the nodes.We use the daily active follower network for this purpose, which is the follower network induced by users who tweet about the impeachment on a given day.
Second, we need to characterize the activity level of the users.We define this to be the tweet rate of the users.We measure the tweet rate as the total number of impeachment tweets posted divided by the duration of our data collection.This gives a stable measure of the posting rate of a user.We use this same posting rate for each daily GHIC calculation as it provides an accurate measure of the general activity level of a user.
Third, we need to know the political sentiment of each user.We obtain this by taking the average sentiment of a user's tweets as measured by our BERT partisanship classifier.
Fourth, we need to identify a subset of users in the daily active follower network as stubborn.GHIC assumes that the stubborn users are not persuadable and their March 28, 2023 28/37 sentiment does not change.We first set all bots to be stubborn, as they are automated accounts that do not respond to persuasion.We identify stubborn users among the humans based on their political sentiment or opinion.Studies have shown that stubborn users have extreme opinions [65].To operationalize this principle, we follow the approach used in [40,43] to define extreme opinions as those below the 10th percentile and above the 90th percentile of the opinions of all users in our dataset.It has been shown that the GHIC is robust to the precise value of these thresholds [40].
Once the requisite information has been obtained, the GHIC of a set of nodes can be calculated.We present the steps for GHIC calculation here, drawing from the presentation in [40].We are given a follower network G = (V, E) with node set V (the Twitter users) and edge set E (the follower edges).We assume each node follows a set of users that we define as their following.Each node v ∈ V posts content (tweets) at a rate λ v .We define the stubborn users as the set V 0 ⊂ V and the non-stubborn users as V 1 ⊂ V .We define Ψ as the vector of stubborn opinions and θ as the vector of non-stubborn opinions.We are given the stubborn opinions Ψ and we obtain the non-stubborn opinions θ by solving where the matrix G is given by the equation: and the matrix F is given by Equation ( 1) is the equilibrium opinions in the opinion dynamics model upon which GHIC is based.

Fig 1 .
Fig 1. Diagram of the different steps taken in our analysis of bots and disinformation in the Twitter discussion surrounding Donald Trump's first impeachment.

Fig 2 ,asFig 2 .
Fig 2. Bot prevalence and activity rates.(left) Fraction of bots and (right) average tweet rate of each account type category.

Fig 3
Fig 3 shows the average media quality score of the different account types.The first

10 − 6 Fig 3 .
Fig 3. Bot language and content analysis.(left) Average media quality score and (right) average toxicity score of tweets posted by different account types.

Fig 3 .
We note that the distribution of toxicity scores for each group is characterized by two clusters located near zero and one.The cluster near one contains the toxic users which set the average toxicity of the group.Further analysis of the distribution of these scores is provided in the Materials and Methods.One striking observation in Fig 3 is that anti-Trump humans have the highest average toxicity score by a wide margin.The next highest toxicity group are pro-Trump humans.Qanon humans and bots have lower toxicity than normal humans (p-value < 10 −6 ).Bots of each partisanship group have lower toxicity levels than their co-partisan human counterparts (p-value < 10 −6 ).The high toxicity levels of anti-Trump humans may be due to their outrage about Trump's actions, and their disagreement with his acquittal.Conversely, low toxicity levels among pro-Trump users may reflect their attempts to post positive content in a show of support for the president.

Finally, we study
bot retweet patterns.A retweet occurs when one account re-posts the content of another, thus sharing the original content with all of their followers.High retweet counts indicate high levels of popularity for the original tweet's content and author.Tables 2 and 3 show the content most retweeted by anti-and pro-Trump bot March 28, 2023 10/37 accounts, respectively, and the number of bots who retweeted each message.These top retweeted messages are primarily authored by elected officials involved in the impeachment proceedings.

Fig 4
Fig 4 shows the accounts most retweeted by anti-and pro-Trump bot accounts, respectively, and the number of retweets each account received from bots.This allows us to examine which accounts benefited most from bot activity.Here we see a clear difference in the structure of the bot retweet distributions.On the anti-Trump side, the bot retweets are distributed in a rather uniform manner over the top ten accounts, with retweet counts ranging from 240,336 to 92,233.However, bot retweet counts on the pro-Trump side range from 765,512 to 113,261, with a very concentrated distribution in which Donald Trump receives the most bot retweets by far,

Fig 5 .
Fig 5. Bot follower network reach and composition.(left) Venn diagram of users who follow anti-Trump and pro-Trump bots.(right) Fraction of co-partisan followers for different bot types.

Fig 6 .
Fig 6.Bot follower network structure.(left) Follower network of bot accounts colored by partisanship, where anti-Trump bots are blue, and pro-Trump bots are red.(right) Follower network of Qanon accounts, where bots are colored red and humans are colored green.

Fig 7 .
Fig 7. Bot impact analysis.(top) Daily generalized harmonic influence centrality (GHIC) score for all bots versus date.(bottom) Boxplot of the daily GHIC per bot for different bot types.

Fig 9 a
histogram of the partisanship scores of all users in our dataset as measured by the classifier.The user scores are obtained by averaging the partisanship score of each user's tweets.

Fig 8 .FrequencyFig 9 .
Fig 8. Confusion matrix of trained partisanship classifier applied to test data.
Pelosi and Dems blew it!Impeachment Sham is Backfiring! 0.98Tweets from testing dataset and their opinion scores assigned by the BERT sentiment classifier.The tweets are ordered by opinion score.An opinion of zero is anti-Trump and an opinion of one is pro-Trump.

Fig 10 .
Fig 10.Bot probability classifications on an example day.Histogram of the bot probabilities calculated by the factor graph algorithm [40] based on the impeachment retweet network on February 1, 2020 (an example day).

Fig 12
Fig 12 shows a violin plot of the mean user toxicity score for the different user types.

Fig 12 .
Fig 12. Violin plot of the mean user toxicity score for different user types.

Table 1 .
Data collection results, by account type.Statistics of account types and tweets in Twitter dataset for the first impeachment of U.S. President Donald Trump.
Fig 5shows the number of unique accounts who follow at least one bot, and the partisanship of the bots they follow.We find that pro-Trump bots have more followers than anti-Trump bots, which may be due to the are co-partisan (i.e.those who share the same political affiliation).A higher value for this measure indicates greater ideological homogeneity in the followers of the bots.Fig5shows the co-partisan fractions for each bot type.We find the values are quite high, being above 0.87 for all bot types.There is a 1% difference in the co-partisan follower fraction between the anti-Trump and non-Qanon pro-Trump bots (p-value < 10 −6 ).However, the Qanon bots have have a co-partisan fraction that is 6% to 7% greater than the non-Qanon bots (p-value < 10 −6 ).This suggests that Qanon bots have an audience that is even more partisan than a standard bot.

Table 6 .
Keywords used to identify pro-Trump users.

Table 7 .
Example partisanship classification scores.Tweet text Opinion Mark my words... Trump is starting a war to distract from the Impeachment.

Table 8 .
Q-anon user profile terms.