Data

Study participants

Since the debate content we study originates from one of the most popular British topical debate programs, we recruit a representative sample of British citizens above the age of 18 using the online platform Prolific. The recruitment time frame ran from June 18th until June 26th, 2024. Participants are informed about the purpose of the study, consent to participate and receive a participation fee for compensation which lies above UK minimum wage. We recruit a total of 948 participants, who are distributed randomly across the different tracks that the study comprises, resulting in at least two judgments for each of the 1560 question/response pairs (520 original question/original response pairs, 520 original question/impersonated response pairs, and 520 pairs of original question/original responses with random speaker) in each track.

Study design

Stylistic comparison

We also measure stylistic differences between the original and the impersonated responses by comparing discourse-related linguistic patterns. This allows us (1) to understand if the responses share properties on the linguistic surface and (2) whether the language is related to human judgments in terms of authenticity, coherence, and relevance. The study is based on the following linguistic properties:

• Syntactic complexity: Syntactic complexity in terms of the mean number of conjuncts, clausal modifiers of nouns, adverbial clause modifiers, clausal complements, clausal subjects and parataxis per sentence as an approximation of language complexity. [18]
• Nominalisations: The number of nominalisations per sentence as an approximation of the of abstractness of the language.
• Modals: The number of modal constructions (e.g., ‘definitely’, ‘potentially’) per sentence as a signal of the stance of the speakers towards their utterances. [19]
• Discourse markers: The number of discourse markers (e.g., ‘first’, ‘moreover’) per sentence as an approximation of the coherence of text and the use of explicit argumentative structure. [20]
• Epistemic markers: The number of epistemic markers (e.g., ‘I think’, ‘in my opinion’) as an indication of the commitment of a speaker to the message they convey.
• Lexical diversity: Lexical diversity measured with MTLD [21] as an approximation of the diversity of the used vocabulary.
• Lexical overlap: The percentage of words in the question (excluding stop words) that also appear in the response as an approximation of the influence of the question on the response.

These stylistic features are automatically extracted from the original and the generated responses using a combination of rule-based, stochastic and neural models for natural language processing. Modals, discourse markers, epistemic markers, nominalisations, and the lexical markers are normalised by the number of sentences within a response.

Statistical analysis

The inter-rater reliability between the two judgments for authenticity, coherence, relevance and content is measured with Cronbach’s [22]. Additionally, we report pair-wise differences between the judgments to quantify the disagreement between participants. We exclude confidence and familiarity because we cannot expect agreement regarding a subjective self-reflection. For the subsequent statistical analysis, we map the Likert scales to the integers [−2, −1, 0, 1, 2] and compute the average rating between the two judgments for the same data point. Since the variables from our survey are based on Likert scales, we use non-parametric rank-based statistical tests.

In Track1 we assess the difference in authenticity, coherence, and relevance between the original responses and the impersonated responses. The track has a between-subjects design (i.e., actual and impersonated responses are rated by different participants) with data that is paired by the question and the speaker. Consequently, we use a two-sided Wilcoxon signed rank test [23] to determine if the difference between both populations (actual versus impersonated responses) is significant.

Track2 uses a within-subjects design (i.e., one participant judges both the actual and the impersonated response). We conduct a two-sided one-sample Wilcoxon signed rank test to determine if the judgments regarding authenticity, coherence, relevance, and content are significantly different from zero. For authenticity, coherence, and relevance, a significant tendency towards negative values means that the participants favour the original responses; a significant tendency towards positive values means that the generated, impersonated responses are favoured. Regarding content, a significant positive value means that the content between original and impersonated responses is similar; a negative value means that the impersonated content is different from the actual content by the speaker. We post-process the data such that the original response is always on the left and the impersonated response is always on the right.

With the data from Track3 we assess if the authenticity and the confidence in the rating depend on whether the speaker is real, random, or impersonated. This results in three populations. The track has a between-subjects design where the populations are paired by the question and the actual speaker. To elicit whether there is any difference between the three populations, we use a Friedman test [24] with a Bonferroni-Dunn post-hoc test based on pair-wise two-sided Wilcoxon signed rank tests. This determines which differences between pairs are significant. Additionally, we use familiarity judgments to understand how this affects authenticity. For this, we conduct a subgroup analysis where we split the ratings into those where the familiarity is less than 0 (i.e., ratings where the participants are not/to a limited extent/fairly familiar with the speaker) and judgments with a familiarity greater than or equal to 0 (speakers are somewhat familiar/familiar with the speaker). For the latter subgroup, we do not have paired data anymore, because we have independent raters for the three populations. For instance, the raters for the responses attributed to the actual speakers may be familiar with different speakers than the raters for the impersonated responses, leading to different subgroups. We therefore use a Kruskal-Wallis test [25] with Bonferroni-Dunn post-hoc tests based on pair-wise two-sided Wilcoxon–Mann–Whitney tests [26].

The statistical analysis of the linguistic surface markers is similar to the analysis of Track1 since we also have two populations for each linguistic marker (actual versus impersonated responses). Since the data from the linguistic markers does not follow a normal distribution (visual analysis of the distribution in Fig 5 shows, for instance, long tails), we also use non-parametric tests, namely two-sided Wilcoxon signed rank tests to determine if differences for each of the seven linguistic markers are significant.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Judgments when a debate question, the name of the speaker, and either the GPT-generated or the actual response by the actual speaker are shown.

Violins show a kernel density estimation of the probability distribution, the miniature box-plots depict the median, upper and lower quartiles, and the whiskers the largest/smallest value observed within 1.5 times the interquartile range of the upper/lower quartile. The statistical markers reported are the p-value of two-sided Wilcoxon signed rank tests, the effect size r, the sample sizes n, mean values M and standard deviations SD.

https://doi.org/10.1371/journal.pone.0347757.g001

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Judgments when a debate question, the name of the speaker, and both the actual and GPT-generated responses are shown side-by-side.

The stacked bar chart reports the percentages of the ratings that we observed. The statistical markers reported are the p-value of a two-sided one-sample Wilcoxon signed rank tests for a difference from zero, the effect size r, the sample sizes n, mean values M and standard deviations SD.

https://doi.org/10.1371/journal.pone.0347757.g002

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Judgments when a debate question with either the response and biography from the actual speaker, the GPT-generated response and the biography of the actual speaker, or the response from the actual speaker but the name and biography of a random public person are shown.

Violins show a kernel density estimation of the probability distribution, the miniature box-plots depict the median, upper and lower quartiles, and the whiskers the largest/smallest value observed within 1.5 times the interquartile range of the upper/lower quartile. The statistical markers reported are the p-value of the omnibus test for differences and pair-wise Bonferroni-Dunn correct two-sided post-hoc tests, the effect size r, the sample sizes n, mean values M and standard deviations SD.

https://doi.org/10.1371/journal.pone.0347757.g003

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Judgments whether the content of the actual response and the GPT-generated response are the same.

The actual and impersonated responses are shown side-by-side. The stacked bar chart reports the percentages of the ratings that we observed. The statistical markers reported are the p-value of a two-sided one-sample Wilcoxon signed rank tests for a difference from zero, the effect size r, the sample sizes n, mean values M and standard deviations SD.

https://doi.org/10.1371/journal.pone.0347757.g004

Thus, we conduct three statistical tests with the data of Track1, four with the data of Track2, three with the data of Track3 and seven statistical tests for the linguistic markers, i.e., a total of 17 tests. We use a conservative approach based on Bonferroni correction [27] to account for multiple tests and consider results as significant if the p-value of a test is less than . Based on the large size of our populations with 520 question/response pairs and assuming that we observe differences of 0.5 points (i.e., half a step on the Likert scales), we compute the expected statistical power as . Consequently, differences of 0.5 or larger in judgment should always be picked up by our tests and if there are no differences in judgments, there is only a 5% chance that we find a difference that is not there.

We report the arithmetic mean (M) and standard deviation (SD) as statistical markers for the populations. While our data is not perfectly normal, it also does not have severe outliers or multimodalities, so we prefer the clear interpretation of the arithmetic mean (M) and standard deviation (SD) to report statistical markers for populations. We use the biserial rank correlation r [28] to measure the effect size. Violin plots visualise the distribution of the data based on a kernel density estimation of the underlying probability distribution. The violins include miniature box plots that depict the median, upper and lower quartiles and the whiskers defined as the largest/smallest observed value at most 1.5 times the inter-quartile range away from the upper/lower quartile. Additionally, we use stacked bar charts to depict ratios of Likert scale items, where appropriate.

Supplemental material S2 Appendix provides details regarding the results, e.g., the demographic information of the participants. Supplemental material S3 Appendix reports on the results of additional quantitative analysis of possible confounding factors, i.e., randomisation of speakers assigned in Track3, the lengths of responses, and the influence of spelling errors. The results for all these factors are negative, i.e., it is highly unlikely that these aspects serve as alternative explanations of our findings.

The statistical analysis of the data is mostly implemented in Python. We use pandas 2.2.2 and numpy 1.26.4 for processing the data, pingouin 0.5.4 for the calculation of Cronbach’s , effect sizes r and pair-wise tests, scipy 1.13.0 for the omnibus tests, and seaborn 0.13.2 for the generation of plots. We compute the statistical power with the R package mkpower 0.9.

Qualitative analysis

The qualitative analysis in this study is two-fold: On the one hand, we code the free-text answers from the exit poll of the survey using inductive coding [29] and have one author assign one or more codes to each answer. The codes are aimed to capture the intent of the free-text answer, e.g., convey the reason for changes in the exit poll or observations regarding the impersonated content that the participants found striking (details on the codes are in supplemental material). This is initially done for twenty answers, at which point the coding is checked by and discussed with a second author, resulting in an agreed-upon coding taxonomy. The first author then continues to code the remainder of the data. Upon completion of this coding, the second author again checks all codes and discusses the coding to achieve agreement in the same manner as for the initial set of codes. We then conduct one round of axial coding [30] to group related codes into categories. Same as above, the axial coding is initially conducted by one author, then checked by and discussed with a second author to achieve agreement. We do not report inter-rater agreements for this data, because all data is fully checked by two authors and all disagreements are discussed and resolved, which results in final coding on which both annotators perfectly agree.

On the other hand, we conduct a qualitative analysis to better understand the human judgments regarding the differences in content between actual and impersonated responses, with a specific focus on whether the stance between actual and impersonated responses differs. For this, we randomly sample 50 pairs of actual and generated responses that were judged to be different in content by the study participants. We use deductive coding to determine if (a) the two responses have the same stance, i.e., arrive at the same conclusion, argue for the same points or take the same side, (b) express a different stance, i.e., draw different conclusions or argue for another position, or (c) whether this distinction is not applicable, e.g., because the question does not require the hearers to take a stance. Additionally, we determine for each response whether it addresses the question. Similar to the coding procedure above, the coding was first done by the one author and then checked by a second author. Disagreement between the two annotators was minimal and adjudicated to achieve agreement.

Ethics statement

Our study used human subjects as participants in a survey. The ethics committee of the University of Passau approved this study (ref. III/Herbold.I-07.5095/240229). The study makes use of an online survey that obtained informed written consent prior to participation in the study.

Impersonated responses are perceived as more authentic, coherent and relevant

The results clearly show that LLM-generated, impersonated content is judged as more authentic, coherent, and relevant than the actual debate responses. When the participants only see one question and its response (either actual or impersonated) (see Fig 1), we observe a significant difference across all three dimensions with a large effect size for authenticity (r = −0.55), relevance (r = −0.82) and coherence (r = −0.84) in the responses. When the participants directly compare an impersonated response with an original response along these dimensions (see Fig 2), the results are supported: The effect sizes for relevance (r = 0.79) and coherence (r = 0.87) remain large, but the difference in authenticity decreases to a weak effect (r = 0.28), so the gap between impersonated responses and actual responses is smaller in this setting (but it is still statistically significant). If the participants see the biographies of the speakers during rating (see Fig 3, top-left), we observe a similar effect size for the authenticity when comparing actual and generated responses than when the questions are shown side-by-side (r = 0.25). This means that the authenticity of the impersonated response is still higher than that of the actual response, even if biographies are provided.

An important control in our study is whether the debate content is just generally assumed to be authentic, instead of only when the response matches the common knowledge that the public has about the speaker. The data in Fig 3 shows that when deliberately assigning an actual response to a randomly picked (wrong) speaker, the authenticity is significantly lower compared to when the response is assigned to the actual speaker with the actual response (r = 0.39) or the actual speaker with the impersonated response (r = 0.59). When we take into account the confidence of the participants in their rating, we do not find any significant differences in the certainty of attributing original or impersonated responses to the actual speaker, or attributing an actual response to a random speaker. If we only consider a subgroup of data where the participants are highly familiar with the speakers, this again neither affects the authenticity nor the confidence: While the significance tests for differences between actual and impersonated responses, as well as actual speakers versus random speakers, are not significant anymore, the distributions are almost exactly the same as for the full dataset. This indicates that there is no shift in distributions, but rather the effects are too small to be detected with the smaller subgroups.

Original content is different from impersonated content

If the authenticity is rated high, but the content of the original response differs from the content of the impersonated response, i.e., the impersonated content is not in line with the actual statements of the person, we are faced with a situation where GenAI can be employed for targeted misinformation about the speaker’s point of view. Our results show that a significant majority of actual responses are judged to be different in content from the impersonated counterparts, though the spread in the distribution is fairly large (see Fig 4). About half of the responses are considered to be dissimilar in comparison to only about one-third of responses that are considered similar. We observe no notable pattern or correlation between the similarity of the content and the authenticity of the responses ().

For the qualitative analysis of the differences, we sample 50 of the total 255 pairs of actual and generated responses for which the content is judged as being dissimilar. Table 1 summarizes the results of the analysis. Three categories emerge: The first category comprises cases where the differences in content are about how arguments are presented, but where both the actual and the generated response arrive at the same conclusion, while also addressing the question (n = 16 for responses with a stance, n = 5 without a stance, 42%). The second category comprises cases where only the actual speaker (n = 2, 4%) or only GPT (n = 14, 28%) responds to the question and the other source dodges the question. We do not annotate these instances as exhibiting a difference in stance – one side does not elaborate on the topic of the question and therefore does not communicate a stance towards it at all, so the identification of the difference in stance is not applicable. Overall, this means in 32% of cases, the question is dodged by one source. The third (and most important category) captures cases where the stance between the actual response and the generated response is notably different (n = 13, 26%). If we extrapolated this to the full dataset, we would expect that around 13% of generated responses communicate a different stance than the actual response.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Results of qualitative analysis of the differences in content.

https://doi.org/10.1371/journal.pone.0347757.t001

Human judgment is reliable

We measure the inter-rater reliability with Cronbach’s [22] for the two judgments on authenticity, coherence, relevance, and content that we get for each question/response pair. The ratings are based on a five-point Likert scale. Additionally, we report the pair-wise differences between the two participants to understand which disagreements our participants have. We exclude confidence and familiarity because we cannot expect agreement regarding a subjective self-reflection. For the subsequent statistical analysis, we map the Likert scales to the integers [−2, −1, 0, 1, 2] and compute the average rating between the two judgments for the same data point. Since the measures from our survey are based on Likert scales, we use non-parametric rank-based statistical tests.

In all variables, we observe an overall modest agreement when measured with Cronbach’s , with values of at most . We analyse the data to understand which combinations of different judgments we observed and find that regarding authenticity (across all variants) there are relatively few polar differences, i.e., one participant rating an item as authentic and the other as not authentic. For relevance, coherence and content the differences are in how positive a judgment is, with small differences of a single point (e.g., ‘neutral’ instead of ‘agreement’), again showing that while the absolute ratings have some variance, the tendency regarding the judgment is typically the same for both participants. Overall, the tendency towards positive or negative judgments about a variable is fairly consistent, especially given our large sample size which is a representative cross-section of the British society.

We note that our method neither allows us to observe participants directly nor measures underlying aspects regarding their capabilities (e.g., their reading comprehension). In principle, if participants with different capabilities are not evenly assigned to tracks, this could lead to unobserved confounding effects. However, given our large sample size, a strong influence of such confounding effects is unlikely.

Linguistic style is different

To provide a perspective different from the human evaluation, we augment our results with a comparison of the linguistic properties of the original and the generated responses (see Fig 5). This experiment yields a number of interesting insights: First, the complexity of the sentences in terms of the number of conjuncts, clausal modifiers, clausal complements, clausal subjects and parataxes as well as the use of modals such as ‘should’ and ‘must’ are not significantly different between the actual responses and the impersonated ones. Secondly, the actual responses contain more discourse markers (e.g., ‘because’, ‘therefore’) than the impersonated responses, even though with a small effect size (r = 0.24). The reason for the statistically significant difference is that there is a long tail of actual responses that contain many discourse markers, even though the peak of the distributions is the same for actual and impersonated responses.

Epistemic markers like ‘I think’ are used substantially more often in original responses – they are only rarely found in the impersonated statements, leading to a large effect size (r = 0.87). The rarity in generated responses is not surprising, giving that these markers indicate that the speaker has a stance on some issue, which we expect to be strongly the case in a broadcast political debate and much less so in the case of an LLM.

Furthermore, the impersonated responses contain more nominalisations (r = −0.88) and have a higher lexical diversity (r = −0.92), both with large effect sizes. The overlap between the words from the question and the response is higher for impersonated than actual responses, with a large effect size (r = −0.98). In fact, the distribution shows that it is not uncommon for all words from the question to appear in the impersonated responses, while this is only rarely true for the actual responses.

Public opinion

Public opinion on the use of AI technology for public debates is collected in two rounds in the exit poll: First, the participants respond without prior knowledge of the data source they just rated (actual versus generated). In the second step, the source of the data is revealed and the participants are asked the same questions on the use of AI technology again.

The results of our exit poll prior to revealing the use of AI (see Fig 6) paint a clear picture: The participants mostly state that they are familiar with AI. Interestingly, while they mostly believe that AI cannot provide valuable contributions to public debates, they simultaneously claim that they support the use of AI if it is made explicit and if it is known how the system was developed. When it comes to the matter of regulating AI, the participants’ opinions are rather mixed, with roughly equal-sized groups favoring regulation, opposing regulation, and being undecided. After revealing the use of AI, over 90% of the participants do not change their opinion. For those who do change their opinion, we see a clear trend: The participants realize they are less familiar with AI than they thought, but also have a more favorable opinion on the use of AI in debates, while at the same time seeing a bigger need for regulation.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Linguistic surface of actual debate responses versus impersonated debate responses.

Violins show a kernel density estimation of the probability distribution, the miniature box-plots depict the median, upper and lower quartiles, and the whiskers the largest/smallest value observed within 1.5 times the interquartile range of the upper/lower quartile. The statistical markers reported are the p-value of two-sided Wilcoxon signed rank tests, the effect size r, the sample sizes n, mean values M and standard deviations SD.

https://doi.org/10.1371/journal.pone.0347757.g005

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Results of the exit poll on the opinion of the participants.

The stacked bar chart reports the percentages of the ratings that we observed. The statistical markers reported are the sample sizes n, mean values M and standard deviations SD. The bar chart depicts the counts for each topic that was addressed in the free-text answers.

https://doi.org/10.1371/journal.pone.0347757.g006

The optional free-text answers (n = 248) further corroborate these results. Many participants explicitly note that they did not change their results (n = 107). However, the other free-text answers indicate that the changes in opinions are caused by the confrontation with the capabilities of AI through the survey. Participants often mention that the impersonated responses are better than the human responses (n = 66) or that the quality of the impersonated responses is higher (n = 32). A few participants noted that the high coherence in the impersonated responses made them sceptical, leading them to believe that AI was possibly used in the survey (n = 7) and that this advantage over the humans can be explained by the live debate setting where the actual panellists do not have time to carefully prepare their responses (n = 5). One participant even notes that this advantage of AI means that AI could be used to train humans for debates. Some participants note that they are not able to distinguish between AI and humans at all (n = 26). There are also a few comments noting negative aspects regarding the quality of the generated responses (n = 4) or that AI was worse than the humans (n = 1), but these are outliers.

Another aspect that is stressed in the comments is the requirement to regulate the use of AI (n = 62), especially with respect to transparency. Particularly, many participants (n = 39) express concerns regarding the potential for deceptive use of AI in debates and the associated risks, some even note feelings of fear, shock, and worry (n = 17). However, some participants express positive emotions like surprise and amazement given the strong capabilities of AI (n = 16). When it comes to the use of AI technology in debates, some participants argue that AI’s convincing performance indicates potential for its use in debates (n = 36), while others question the general concept of AI debaters (n = 23). For instance, participants are uncertain about AI’s ability to represent party opinions and are concerned that involvement of AI will undermine the value of debates as forums for meaningful discussion between people.