The threat of deepfakes.

There is an abundance of literature speculating about the harms that deepfakes could wreak upon society. For example, Albahar and Almalki [7] suggest that deepfakes could make fake news and misinformation more widely present in all forms of media, leading to citizens being unable to differentiate between real and misleading information. Indeed, some research has shown that individuals cannot discern real videos from deepfakes at a rate better than chance [20]. Just as worryingly, Kobis et al. [20] also showed that individuals overestimate their ability to discern deepfakes from real videos, possibly further increasing their susceptibility to deepfake deception. In a legal context, the validity of images and videos for use as evidence in courtrooms may be in jeopardy because video evidence, which is often considered the gold standard for proof, can no longer be trusted. In a political context, deepfakes could be used as a form of sabotage and defame candidates from opposing parties as you could create a deepfake depicting the opposition candidate saying something derogatory. These concerns are repeatedly highlighted in published literature [14,17,19,21]. Chesney and Citron [18] further discuss that deepfake technology could distort democratic discourse, manipulate elections, and undermine public safety. Concerns around deepfakes have also spread to military and national security risks where, according to Chesney and Citron [18], deepfakes “have utility as a form of disinformation supporting strategic, operational, or even tactical deception” (p. 1783).

Deepfakes may also cause harm in other ways. Firstly, some authors argue that deepfakes could give rise to a “Liar’s Dividend”, where factual information from media and news in society is invalidated and incorrectly labelled as fake news [18]. As an example, a politician could deny the veracity of an incriminating video of themselves by declaring it as a deepfake. This could serve to undermine citizen’s trust in any information they receive regardless of authenticity. Similarly, deepfakes may cause “epistemic pollution” where knowledge of the existence of deepfakes may cause individuals to accuse real videos of being deepfakes [14]. For example, Twomey et al. [22] explored discourses surrounding the Russo-Ukrainian war on Twitter. Surprisingly, the most frequent form of deepfake-related misinformation found in the dataset was real media being labelled as being deepfake Secondly, deepfakes may cause reputational harm, even in situations where the videos have been identified as inauthentic, such as with the generation of deepfake revenge pornography [23]. The reputational damage comes from being depicted in realistic-looking sexually suggestive positions, meant to embarrass and humiliate the victims [23,24]. Importantly, the warnings seen so frequently in literature about deepfakes are based on the assumption that deepfakes can successfully influence the beliefs, memories, and behaviours of viewers. However, there is no existing review that assembles and synthesises the available evidence, to determine whether that view is supported empirically.

Misinformation effects on beliefs, memories, and behaviour.

Decades of research have demonstrated that encountering misinformation can affect people’s beliefs, with more recent research focused on fake news stories. Exposure to fake news is correlated with a range of adverse outcomes. For example, Ognyanova et al. [25] demonstrated that false information is linked to reduced trust in mainstream media, while Balmas [26] showed that subjection to political misinformation is associated with attitudes of inefficacy, alienation, and cynicism towards political candidates. Misinformation in a health context can also influence people’s beliefs. For example, studies have shown that misinformation about COVID-19 during the pandemic has been associated with increased vaccine hesitancy [27–29]. Beyond text-based fake news stories, research has shown that doctored photographs and other forms of manipulated media can affect beliefs as well. For example, Hameleers et al. [30] demonstrated that a combination of doctored photos and misinformation texts was more credible than text alone, suggesting that visual media could enhance the potency of misinformation. In agreement with this finding, Nash et al. [31], discovered a small but consistent effect of doctored photos on people’s beliefs about prior events, as well as the potential for doctored videos to distort participants’ beliefs about actions they had performed [32,33]. Messaris & Abraham [34] suggests that visual information provides an evidential quality that enhances its credibility over text. The multi-modal audio-visual aspect of deepfakes may suggest an enhanced potency at generating and propagating misinformation. Current literature has also supported this sentiment, with researchers speculating that the visual aspect of deepfakes makes them more likely to affect our beliefs [35,36].

A large body of evidence has also demonstrated clear effects of misinformation on memory. Elizabeth Loftus first established “the misinformation effect” where post-event misleading information can alter one’s memories of an event [37]. The misinformation effect can be seen in political settings where false memories were induced by fake news stories for the abortion referendum in Ireland [38] and Brexit [39]. Similarly, there is also evidence to support the distortion of memory due to exposure to doctored photos. Wade et al. [40] induced false memories of a childhood hot air balloon ride by showing participants doctored photos. This effect was replicated by Sacchi et al. [41] when they found that doctored photos caused participants to misremember elements of important historical events such as the 1989 Tiananmen Square protest in Beijing and the 2003 Rome anti-war protest. Many researchers have proposed that deepfakes will similarly induce vivid false memories [35,36,42]. However, Greene et al. [43] and Garry and Wade [44] found that visual-based misinformation in the form of photographs did not induce false memories better than simple text narratives. They reasoned that the narratives encouraged deeper processing of misinformation, leading to more significant distortions in memory compared to photographs. Many previous studies, including Wade et al. [40] and Saachi [41] did not include text-based misinformation as a comparator, so though they reported substantial rates of false memories, these may not have been due to the doctored images. Thus, it is difficult to predict whether deepfakes might be more potent at manipulating memory than non-technical means of conveying misinformation.

There are significantly fewer studies on the effects of misinformation on behaviours. Research on the COVID-19 pandemic has shown that misinformation exposure from social media has induced a decline in intent to be vaccinated [45]. Similarly, a decline in childhood vaccination rates against measles, mumps, and rubella (MMR) was found to be related to misinformation about the MMR vaccine [46]. Greene and Murphy [3] experimentally quantified the effects of misinformation exposure on behavioural outcomes and found a small but significant effect of COVID-19 misinformation on behavioural intentions. Conversely, de Saint Laurent et al. [29] found no effect of COVID-19 misinformation on behavioural intentions. The existing literature focuses on health contexts and misinformation delivered through text only with little agreement, thus it is unclear how misinformation affects behavioural intentions in different contexts and mediums. However, general claims have been made in the literature about the ability of misinformation to modify and manipulate behaviour, threaten individual autonomy, and jeopardize democracy [47].

Factors influencing the effects of misinformation on people’s beliefs, memories, and behaviour.

There is a wealth of research investigating the individual factors that lead people to be influenced by inaccurate or false information. For example, people who engage in analytical reasoning and reflection are less likely to be deceived by misinformation [48,49]. Pennycook and Rand [5] and Bronstein et al. [50] also mirror these findings, showing the negative correlation between cognitive reasoning and misinformation susceptibility. This shows that the tendency to engage in careful analysis rather than the intuitive acceptance of information is important in discerning truth from misinformation. This idea has then been used to design interventions to help combat misinformation. For example, misinformation “inoculation” or “pre-bunking” is a strategy which forewarns individuals of misinformation by providing accurate information on the misinformation content they are about to view. This strategy works by provoking individuals to critically analyse the content they are about to view, bypassing simple intuitive acceptance of information in favour of a purposeful engagement with the information [51]. Despite the wealth of literature investigating these individual factors and their implications, very few of these studies have specifically investigated deepfakes. Due to the multi-modal format of deepfakes, it is not known whether deepfakes are more, less, or equally effective in delivering misinformation and manipulating people’s beliefs, memories, and behaviours compared to misinformation presented as text or photos, despite many claiming this to be the case [35,36]. Also, it is evident that factors such as cognitive style can influence the effects of misinformation on beliefs, memory, and behaviour and it would be reasonable to assume that deepfake misinformation would follow closely with this literature. The current scoping review will provide a clearer picture regarding the impact of deepfakes. be the first step to establishing the effects of deepfakes in misinformation literature.

RQ1. Is there evidence to suggest that deepfake exposure has a measurable impact on people’s beliefs, memories, or behaviour?.

Our review yielded eight studies that examined how deepfake exposure does have a measurable effect on people’s beliefs across four research topics. Firstly, experiments by Hwang et al. [56] and Lee and Shin [57] found that participants rated misinformation messages presented as a deepfake as more vivid and credible compared to text and text-photo formats, suggesting deepfakes may be a uniquely persuasive form of misinformation. Secondly, Hameleers et al. [58] and Dobber et al. [59] document the effects of deepfakes on beliefs in a political context by exposing participants to deepfakes of political figures saying disrespectful [59] and politically incongruent [58] statements. Their results show that deepfakes negatively impact people’s attitudes towards the depicted politicians, even if the deepfake was not rated as credible [58]. Thirdly, Vaccari and Chadwick [36] and Ahmed et al. [60] show that deepfakes can also be harmful by influencing our beliefs about authentic information, promoting mistrust in news and social media [36], as well as affecting our perceptions of real information about public figures by retroactively showing participants deepfakes of those public figures [60]. Finally, studies by Wu et al. [61] and Lu and Chu [62] investigated how deepfakes influence our beliefs in a potentially positive way. Wu et al. [61] demonstrated how viewing deepfakes of oneself on the body of a celebrity positively impacts young women’s feelings of satisfaction with their own attractiveness and self-image while Lu and Chu [62] explores the impact of so-called “digital resurrections” of deceased individuals and how they could be used advocate for issues related to their cause of death. However, they found that individuals considered these resurrections as disrespectful and demonstrated reduced support for these issues. While there is evidence that deepfake exposure has a measurable effect on beliefs, the varied research questions and relatively low number of studies in each research topic makes it difficult to identify obvious trends in the literature.

Two studies in the review investigated how deepfakes affect memory and potentially create false memories. Murphy and Flynn [4] found that deepfakes can indeed affect people’s memories of events by creating false memories for events that did not occur, but this effect does not seem to be significantly different than other formats of misinformation such as text-only or text with a photo. Similarly, Murphy et al. [63] found that deepfakes were capable of inducing false memories in participants in an entertainment context when this technology is used to replace the faces of actors in films with other famous actors. Interestingly, deepfakes were no more effective at inducing false memories than simple textual misinformation. This finding is tentative and based on just two studies, but it suggests that while deepfakes can manipulate memories, concerns about our memories being uniquely affected by the rise of deepfake misinformation [35] may be overblown.

Finally, we identified four studies that indirectly explored how attitudes and behavioural intentions were impacted by deepfakes. Lee and Shin [57] and Hwang et al. [56], found that people perceived deepfaked messages as more credible and vivid compared to other formats of misinformation, such as text-only and text-photos. They also found that this increase in vividness and credibility was associated with elevated levels of engagement and sharing intentions. Similarly, Ahmed [64] found that individuals are more likely to share deepfakes online when they deem the deepfake to be credible, and Ahmed and Chua [65] found that deepfakes, compared to cheapfakes or audio deepfakes, are especially likely to be shared. Noticeably absent from these few studies of deepfakes on behaviour are studies on a wider variety of behaviours beyond deepfake sharing (e.g., vaccine uptake, voting choice, etc.) and any studies assessing real-life behaviours or behaviours over a longer time period. This echoes gaps in misinformation research more generally and is not unique to deepfake research [29]. Evidently, the current empirical literature examining the effects of deepfakes on beliefs, memory, and behaviour is insufficient in number to draw strong conclusions. While the initial studies suggest a measurable effect of deepfakes on beliefs, memories, and behaviours, the relatively low number of studies, the varied research findings, inconsistent deepfake stimuli, and lack of behavioural research highlight many unanswered questions.

RQ2. Is there evidence to suggest that the effects of deepfakes on beliefs, memories, and behaviour are different from pre-existing forms of misinformation?.

As shown in Table 1, we found just six studies that compared the deceptive potential of deepfakes to other misinformation formats such as text or text-photos [4,56,57,63,66,67]. These six studies demonstrated mixed results. For example, Murphy and Flynn [4] and Murphy et al. [63] both compared misinformation presented as deepfakes and as text to participants and found that deepfakes did not produce significantly different effects than the text conditions in inducing false memories in participants. Hameleers et al. [66] and Hameleers [67] found that deepfakes were not perceived as more credible than misinformation presented as text. In direct contrast to this, Lee and Shin [57] and Hwang et al. [56] compared deepfakes to misinformation text and texts with photos and found that misinformation messages presented as deepfakes were rated as more credible and vivid. Thus, the very sparse evidence to date suggests that at least when we assess memory distortion or credibility, there is no clear advantage for deepfakes relative to existing forms of misinformation. Despite much speculation as to the grave threats posed by deepfake technology and how much more potent deepfake misinformation is relative to other formats [7,19,35], we simply do not have the data and sufficient agreement of evidence to support these claims.

RQ3. Is there evidence of social, psychological, or technical factors that influence the effects deepfakes have on beliefs, memories, and behaviour?.

13 studies provided evidence of psychological, social, or technical factors which influence how deepfake exposure affects our beliefs, memories, and behaviours. Seven of these studies highlighted cognitive ability or a factor related to cognition as a factor that may influence the impact of deepfake exposure. Most of these studies considered cognitive ability as the ability to think analytically and be able to critically assess information or stimulus past simple intuition [5]. Ahmed [68], Ahmed [69], Ahmed et al. [70], and Ahmed and Chua [65] reported that individuals with higher cognitive ability were less likely to self-report the inadvertent sharing of deepfakes. This was measured by directly asking participants whether they had recently shared a deepfake and later discovered that the video was a hoax. This is problematic as it relies on participants’ own subjective interpretation of what deepfakes are and does not allow the reporting of deepfake sharing that went unnoticed. Individuals with higher levels of cognitive ability are less likely to consider deepfakes they see as credible or accurate [58,60] Finally, Iacobucci et al. [71] found that individuals with low levels of bullshit receptivity (the tendency of an individual to believe baseless claims) and primed with knowledge about deepfakes were more likely to recognise that a video was a deepfake. This experiment measured deepfake recognition by asking participants whether a clip of a face-swapped actor in the movie “The Shining” was similar to the original scene due to the actor’s acting ability, or digital video-editing technology.

Five studies found that an individual’s political interests, views of politicians, and ideology can also affect how deepfakes are interpreted and consumed [66,68,72–74]. Hameleers et al. [66] found that deepfake disinformation that was ideologically congruent was perceived as more credible than disinformation that was ideologically incongruent – however this was equally true for text-based disinformation, so does not suggest anything unique to deepfakes. Sharma et al. [74] found that political brand hate (an individual’s hatred towards a political party or politician due to who or what they represent) was associated with sharing deepfakes which represent the ideologically incongruent party or “brand” in poor light. Similarly, Lee and Shin [72] found that individuals were more likely to believe and intend to share deepfake news which advocated for positions congruent with their own beliefs than individuals with incongruent beliefs. Regardless of political stance or position agreement, Ahmed [68] reported that individuals who simply have an interest in politics are more likely to inadvertently share any deepfakes they encounter. However, this finding is unreliable due to the aforementioned issues of interpretation, self-reporting, and measurement of deepfake sharing behaviours. On a separate note, Ng [73] found that an individuals’ susceptibility to deepfake videos of a politician depends on the perceived trustworthiness and dangerousness of that politician. If an individual perceived a politician to be dangerous, then they are less likely to regard deepfake videos of them as authentic, and deepfakes of perceived trustworthy politicians are more likely to be regarded as authentic.

Finally, our search yielded one study which identified social factors which influenced how individuals perceived the credibility and authenticity of deepfakes. Jin et al. [75] presented participants with a YouTube video, where they manipulated the number of views the video had (representing the popularity of the video) and the number of followers the channel had (representing the trustworthiness of the source). They found that more views increased the perceived credibility of the video, and more followers increased perceived authenticity of the video. This study highlights how individuals have a tendency to perceive deepfakes as more credible and authentic if others appear to do so too. Once again though, the lack of studies on this topic limits the generalizability of these results. Notably, there were no studies exploring technical factors which influence perceptions of deepfakes, such as the quality or resolution of the deepfake, clarity of audio, visual artifacts, or realism of the deepfake.

RQ4. What kind of deepfake technology is being used in current empirical deepfake research?.

While not all the deepfakes used in the included studies were available for other researchers to view, it is clear that they varied considerably in terms of resolution, audio quality, accuracy of lip-syncing to voice, level of realism of the faces, and visual artifacts due to deepfake technology. As detailed in Table 1, they were taken from multiple different pre-made sources such as from the Canny AI art installation [4,56,60,64,70] and YouTube [4,36,71,75] or they were produced by the researcher themselves using software such as the ZAO mobile app [61], Faceswap software [57], custom made by a VFX artist or similar professional [58,66,67,74] or Tacotron 2 software [59]. The deepfakes included in this review that were drawn from the Canny AI art installation or YouTube could potentially be of a higher quality and more realistic since they were professionally produced by a team (Canny AI or Buzzfeed), but the downside to these studies is the lack of experimental control over the deepfakes used. The benefits for using pre-made deepfakes are that they enhance ecological validity as these deepfakes were made outside of an experimental setting, allow for comparison of results due to identical deepfake stimuli, and are freely available for other researchers to use to replicate the study allowing others to critically evaluate the work. Deepfakes which were produced by the researchers could be tailored towards answering specific research questions such as in Lee and Shin [57] or Dober et al. [59], where they manipulated specific faces or facial features and voices respectively rather than being constrained to designing experiments around pre-existing deepfakes. However, custom deepfakes give rise to issues of reporting as they are often not made readily available online [57,59]. This may be for ethical reasons due to researchers not wanting to contribute to the spread of misinformation online by uploading their custom deepfake, specifically if the deepfake was political in nature.

With deepfakes primarily being an audio-visual form of misinformation, their perceived credibility determines the quality of the deepfake. Only four studies from the 20 experimental studies in this review reported assessing the perceived credibility of the deepfake stimuli in their experiments as an indicator for quality [4,59,66,74], while six other studies assessed credibility as a dependent variable only [56,58,62,67,73,75]. Without assessing perceived credibility of deepfakes, it is difficult to interpret null results and assess video quality. Among the studies that reported assessing perceived credibility, Sharma et al. [74] asked a group of students (separate to the study sample) before the study to distinguish between an authentic video and a deepfake. Dobber et al. [59] and Murphy and Flynn [4] reported asking participants at the end of the study to rate the deepfakes they viewed on how authentic or credible the participant thought the deepfakes were, and Hameleers et al. [66] assessed credibility indirectly by comparing the perceived credibility and realism scores of statements made in authentic videos to deepfaked ones.

Of particular note, an overarching pattern seen across the four research questions is the short-term nature of the outcome measures in all of the studies included in this review. None of these studies included any kind of longer-term follow-up, instead all assessed beliefs, memories, or behavioural intentions instantly after the deepfake was viewed. Perhaps of more importance is the fact that most of the included studies did not include any comparator, so could not investigate what it is about deepfakes that may be novel or particularly efficacious at affecting beliefs, memories, and behaviours. The research indicates that while deepfakes may sometimes have a measurable impact on beliefs, memories, and behaviours, it is unclear if these effects are lasting or indeed any greater than effects of less technologically sophisticated forms of misinformation.