ChatGPT- versus human-generated answers to frequently asked questions about diabetes: A Turing test-inspired survey among employees of a Danish diabetes center

Adam Hulman; Ole Lindgård Dollerup; Jesper Friis Mortensen; Matthew E. Fenech; Kasper Norman; Henrik Støvring; Troels Krarup Hansen

doi:10.1371/journal.pone.0290773

Peer Review History

Original SubmissionMay 2, 2023
19 Jun 2023 Decision Letter - Jafar Kolahi, Editor Transfer Alert This paper was transferred from another journal. As a result, its full editorial history (including decision letters, peer reviews and author responses) may not be present. PONE-D-23-12539ChatGPT- versus human-generated answers to frequently asked questions about diabetes: a Turing test-inspired survey among employees of a Danish diabetes centerPLOS ONE Dear Dr. Hulman, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Aug 03 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Jafar Kolahi Academic Editor PLOS ONE Journal requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information. 3. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. In your revised cover letter, please address the following prompts: a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent. b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. We will update your Data Availability statement on your behalf to reflect the information you provide. Additional Editor Comments: The use of chatbots in healthcare has become increasingly popular in recent years. These automated systems are designed to provide patients with quick and easy access to medical information and support. However, for patients with diabetes, the use of chatbots can be hazardous. Diabetes is a chronic condition that requires careful management to prevent complications. Patients with diabetes need to monitor their blood sugar levels regularly, make dietary adjustments, and take medication as prescribed by their doctor. Chatbots may not be able to provide accurate and personalized advice on these matters, which can be dangerous for patients. Furthermore, chatbots may not be equipped to handle emergencies or urgent situations that require immediate medical attention. Patients with diabetes may experience sudden drops in blood sugar levels, which can be life-threatening if not treated promptly. In such cases, relying on a chatbot for help can delay necessary medical intervention. However, while chatbots have their benefits in healthcare, they may not be suitable for patients with diabetes. Patients with this condition should consult with their healthcare provider for personalized advice and support to manage their condition safely and effectively. Hence, please add a paragraph to the discussion section about the hazards of using AI for diabetic patients. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Partly Reviewer #2: Partly ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: No ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors describe a ChatGPT Turing Test to determine whether participants would be able to discriminate whether answers to questions about diabetes were produced from humans or generated from ChatGPT. They secondarily hypothesized that participants with ChatGPT experience would be better at identifying ChatGPT produced answers. Participants were employees at the Steno Diabetes Center Aarhus. Q1: As the authors state, human answers were taken directly from source websites/materials. How do you account for possible participant familiarity/memory of these materials, which could identify human responses? Q2: How do the authors account for different levels of education among this participants with respect to diabetes, and could this affect responses? Q3: Given that the survey was web based, and some of the materials are web based, how do you account for participants performing an internet search for the questions, and finding the exact or near exact answers online? Q4: Given the occasional confabulation by ChatGPT, is it possible that experts on diabetes who took the survey were able to spot these errors to identify the ChatGPT answer? The authors identify 2 examples of this in their survey. Reviewer #2: Thank you for your interesting and rigorous work. The manuscript is well written and the statistical approach is technically sound. The question is relevant to health care practice. Suggestions to improve the manuscript are provided below. 1) The authors do not provide publicly available data due to institutional (Aarhus University Hospital) data use policy and concerns about participants' privacy. It needs to be evaluated if this paper could be considered an exception to PLOS Data policy. 2) Consider adding further characterisation of study participants regarding professional roles (administrative staff? nurses? doctors? other health professionals?). 3) It is stated that the utility of the answers was assessed by the authors and was not part of the survey. This analysis would be relevant to present, specially if conducted (even if post-hoc) by a blinded outcome assessor. It is stated that 2 AI-generated answers included incorrect information, but there is no correctness information regarding human-generated answers. The utility analysis could include correctness, as this would be relevant to the study's aim (to investigate ChatGPT’s knowledge in the diabetes domain). 4) There is no mention to conformity to STROBE guidelines, as required by PLOS One Submission Guidelines for observational studies. 5) The prompts and examples given to ChatGPT could bias the study in favour of non-inferiority. Although the rationale for the prompts is clear, consider stating it as a limitation in the Discussion section. 6) It is stated in the abstract that "Participants could distinguish between ChatGPT-generated and human-written answers somewhat better than flipping a fair coin. However, our results suggest a stronger predictive value of linguistic features rather than the actual content.". Consider making it clearer in the conclusion that the results did not support your initial hypothesis, as stated in the study protocol (we hypothesized that participants who have at least some and up to expert knowledge about diabetes, will not be able to distinguish between answers written by humans and generated by AI in response to diabetes-related questions), and that the non-inferiority margin was reached. I am unsure if the results strongly support the stronger predictive value of linguistic features (considering the AI's incorrect statements), and would suggest keeping this sentence in the Discussion, but retracting it from the abstract. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0290773.r001
Revision 1
24 Jul 2023 Author Response Additional Editor Comments: The use of chatbots in healthcare has become increasingly popular in recent years. These automated systems are designed to provide patients with quick and easy access to medical information and support. However, for patients with diabetes, the use of chatbots can be hazardous. Diabetes is a chronic condition that requires careful management to prevent complications. Patients with diabetes need to monitor their blood sugar levels regularly, make dietary adjustments, and take medication as prescribed by their doctor. Chatbots may not be able to provide accurate and personalized advice on these matters, which can be dangerous for patients. Furthermore, chatbots may not be equipped to handle emergencies or urgent situations that require immediate medical attention. Patients with diabetes may experience sudden drops in blood sugar levels, which can be life-threatening if not treated promptly. In such cases, relying on a chatbot for help can delay necessary medical intervention. However, while chatbots have their benefits in healthcare, they may not be suitable for patients with diabetes. Patients with this condition should consult with their healthcare provider for personalized advice and support to manage their condition safely and effectively. Hence, please add a paragraph to the discussion section about the hazards of using AI for diabetic patients. Thank you for your comments on our manuscript. We aimed to address safety issues in the manuscript from a broader clinical perspective (paragraph #6-7 in the Discussion), but following the suggestion of the Editor, we added a more diabetes and chronic disease specific sentence to paragraph #6. ‘This type of misinformation can have serious consequences to the patients’ wellbeing. However, ChatGPT’s answer also included the importance of monitoring blood glucose levels before, during and after training, which reduces the risk of potential harm. Nevertheless, this example demonstrates the importance of built-in safeguards in AI-based applications. The role of chatbots in the management of diabetes and other chronic conditions, given their inherent complexity and the constant requirement for highly-personalised decision-making on the part of patients and their carers, remains unclear and deserves deeper scrutiny.’ We believe that the rigorous planning of our study sets an example for future studies to evaluate the hazards of large language models applied in healthcare. Reviewer #1: The authors describe a ChatGPT Turing Test to determine whether participants would be able to discriminate whether answers to questions about diabetes were produced from humans or generated from ChatGPT. They secondarily hypothesized that participants with ChatGPT experience would be better at identifying ChatGPT produced answers. Participants were employees at the Steno Diabetes Center Aarhus. We appreciate the reviewers' comments and questions. Please find our point-by-point answers below. Q1: As the authors state, human answers were taken directly from source websites/materials. How do you account for possible participant familiarity/memory of these materials, which could identify human responses? The majority of the questions were from the Danish Diabetes Association’s website, more specifically the Frequently Asked Questions page, as described in the study protocol and the manuscript. ‘Eight questions were identified among the ‘Frequently Asked Questions’ on the website of the Danish Diabetes Association (diabetes.dk, accessed on Jan 10, 2023), an interest organization for people living with diabetes, and the largest patient association in Denmark.’ The Danish Diabetes Association is a patient organization and as such the main target audience of the FAQ page is patients. Therefore, we don’t believe that familiarity with the materials could have a major impact on the results, but of course we cannot exclude this either. We added the following sentence to the paragraph on limitations of our study. ‘Although, most questions and corresponding human-written answers were identified on a website with people with diabetes as their main target audience, we cannot exclude the possibility that some participants were familiar with the sources and recognized the human-written answers.’ Q2: How do the authors account for different levels of education among this participants with respect to diabetes, and could this affect responses? We asked participants the simple question whether they have contact with patients as healthcare professionals, which is the closest proxy for different levels of knowledge about diabetes. We compared these groups with regard to the outcome, which is reported in the Results section and on Figure 1. We agree with the reviewer in that this could definitely affect the outcome, however, many participants were researchers with extensive knowledge about diabetes even without patient contact, which could have diluted the differences. It is impossible for us to investigate this further, as we cannot assess retrospectively the participants’ knowledge about diabetes. Q3: Given that the survey was web based, and some of the materials are web based, how do you account for participants performing an internet search for the questions, and finding the exact or near exact answers online? On the opening page of the survey, participants were asked not to use any help (e.g. Google) or too much time on the answers, but rather their intuition. Therefore, we think the risk of this having a major impact on the results is minimal. Q4: Given the occasional confabulation by ChatGPT, is it possible that experts on diabetes who took the survey were able to spot these errors to identify the ChatGPT answer? The authors identify 2 examples of this in their survey. The correct answer with regard to gestational diabetes was revealed by 63% of participants (68% among healthcare professionals [HCPs], 57% among others), while 47% chose the correct one about type 1 diabetes and physical activity (56% among HCPs, 35% among HCPs). Reviewer #2: Thank you for your interesting and rigorous work. The manuscript is well written and the statistical approach is technically sound. The question is relevant to health care practice. Suggestions to improve the manuscript are provided below. We thank the reviewer for the positive comment and appreciate the suggestions. Please find our point-by-point comments below. 1) The authors do not provide publicly available data due to institutional (Aarhus University Hospital) data use policy and concerns about participants' privacy. It needs to be evaluated if this paper could be considered an exception to PLOS Data policy. We have added more information on this issue in our cover letter. According to our current agreement, the Legal Department of the Region of Central Denmark requires approval from the Danish Data Protection Agency for third parties to get access to the data, which makes it infeasible to share the dataset publicly. We have addressed this issue in the cover letter of the revision. 2) Consider adding further characterisation of study participants regarding professional roles (administrative staff? nurses? doctors? other health professionals?). It is a valid point by the reviewer, but unfortunately, our agreement with the Legal Department does not allow us to link the dataset to any administrative database about the participants. 3) It is stated that the utility of the answers was assessed by the authors and was not part of the survey. This analysis would be relevant to present, specially if conducted (even if post-hoc) by a blinded outcome assessor. It is stated that 2 AI-generated answers included incorrect information, but there is no correctness information regarding human-generated answers. The utility analysis could include correctness, as this would be relevant to the study's aim (to investigate ChatGPT’s knowledge in the diabetes domain). Thank you for pointing this out. We meant correctness instead of utility, as the latter heavily depends on the context and therefore is difficult to assess, which we now revised in the Methods section. All human-written answers were considered correct. 4) There is no mention to conformity to STROBE guidelines, as required by PLOS One Submission Guidelines for observational studies. Instead of the STROBE guidelines, we applied the CHERRIES reporting guidelines. This is specifically developed for web-based surveys and is also part of the Equator Network. 5) The prompts and examples given to ChatGPT could bias the study in favour of non-inferiority. Although the rationale for the prompts is clear, consider stating it as a limitation in the Discussion section. Few-shot learning is a common approach in extracting and assessing knowledge encoded in large language models (Singhal et al. 2022, arxiv.org/abs/2212.13138). The main rationale behind using it was to get answers in a similar format (length, complexity of language) to those of human answers. However, we acknowledge the difference between knowledge encoded in a large language model, and the knowledge an average user can extract from ChatGPT via its user interface. To clarify this issue raised by the reviewer, we added the following sentence to the Discussion: ‘Moreover, an average user might not be able to extract the same level of knowledge from ChatGPT, as we did in an experimental setting using few-shot prompting.’ 6) It is stated in the abstract that "Participants could distinguish between ChatGPT-generated and human-written answers somewhat better than flipping a fair coin. However, our results suggest a stronger predictive value of linguistic features rather than the actual content.". Consider making it clearer in the conclusion that the results did not support your initial hypothesis, as stated in the study protocol (we hypothesized that participants who have at least some and up to expert knowledge about diabetes, will not be able to distinguish between answers written by humans and generated by AI in response to diabetes-related questions), and that the non-inferiority margin was reached. I am unsure if the results strongly support the stronger predictive value of linguistic features (considering the AI's incorrect statements), and would suggest keeping this sentence in the Discussion, but retracting it from the abstract. Thank you for this suggestion. We have revised the abstract to make the main results more clear. ‘Overall, participants could identify ChatGPT-generated answers 59.5% (95% CI: 57.0, 62.0) of the time, which was outside of the non-inferiority zone. … Participants could distinguish between ChatGPT-generated and human-written answers somewhat better than flipping a fair coin, which was against our initial hypothesis. However, our results suggest a stronger predictive value of linguistic features rather than the actual content.’ Attachments Attachment Submitted filename: rebuttal-letter-PONE-D-23-12539_R1.pdf https://doi.org/10.1371/journal.pone.0290773.r002
16 Aug 2023 Decision Letter - Jafar Kolahi, Editor ChatGPT- versus human-generated answers to frequently asked questions about diabetes: a Turing test-inspired survey among employees of a Danish diabetes center PONE-D-23-12539R1 Dear Dr. Hulman, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Jafar Kolahi Academic Editor PLOS ONE Additional Editor Comments (optional): . Reviewers' comments: https://doi.org/10.1371/journal.pone.0290773.r003
Formally Accepted
23 Aug 2023 Acceptance Letter - Jafar Kolahi, Editor PONE-D-23-12539R1 ChatGPT- versus human-generated answers to frequently asked questions about diabetes: a Turing test-inspired survey among employees of a Danish diabetes center Dear Dr. Hulman: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Jafar Kolahi Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0290773.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .