Influences of study design on the effectiveness of consensus messaging: The case of medicinal cannabis

Asheley R. Landrum; Brady Davis; Joanna Huxster; Heather Carrasco

doi:10.1371/journal.pone.0260342

Peer Review History

Original SubmissionJuly 6, 2021
12 Oct 2021 Decision Letter - Lucy J Troup, Editor PONE-D-21-22069Influences of Study Design on the Effectiveness of Consensus Messaging: The Case of Medicinal CannabisPLOS ONE Dear Dr. Landrum, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Nov 26 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Lucy J Troup, Ph.D Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. We note that Figure 1 in your submission contain copyrighted images. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright. We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission: 1. You may seek permission from the original copyright holder of Figure 1 to publish the content specifically under the CC BY 4.0 license. We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text: “I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.” Please upload the completed Content Permission Form or other proof of granted permissions as an "Other" file with your submission. In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].” 2. If you are unable to obtain permission from the original copyright holder to publish these figures under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. Additional Editor Comments (if provided): Thank you so much for your patience. I would like to concur with the reviewers comments that this is a well written manuscript. It could however benefit from some minor revisions. Please look careful at the reviewers suggestions and I look forward to receiving your revised manuscript. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: This was a well written and organized manuscript that explored the effectiveness of consensus and evidence based messaging in view points on medical cannabis. Overall the authors found that a message reporting a high percentage of scientists agreeing is a more convincing than reporting existence of evidence. The researchers also found exposure effects related to research method. This is an important article as it can help media and researchers effectively get their message out and for researchers to choose the best method of testing method effectiveness. It was not clear in the introduction as to why medical cannabis was the topic of choice. The authors noted that "we are less concerned with increasing public support for medical cannabis than we are curious about the persuasiveness of different messaging strategies." which is fine, but this manuscript will attract readers that are concerned with this, therefore I think a bit more needs to be discussed on the topic. My major concern with the manuscript was the control condition. In Figure 1 the two experimental conditions state the "cannabis is effective" while the control condition mentions CBD and the fact that is has no psychoactive effects. I am not sure if that is a good control condition to compare against the experimental conditions, since it is only mentioning CBD when the questions asked were about cannabis. What I would like to see is a discussion on how this issue may have affected the results. Minor things The resolution for the figure was to poor to read them comfortably. In the conclusion, please add a statement on study design your findings. Reviewer #2: Development of concept using different messaging strategies is very interesting, well explained and referenced. Excellent range of testing used, interesting and statistically relevant outcome reported. Some very interesting findings in regards to a shift in perception post consensus messaging, and this would benefit from further exploration in other research papers. This paper has implications beyond the cannabis field and is an excellent contribution to the topic. One point that needs work on: Overarching research question at the start of the paper is whether consensus message or evidence messaging influenced the perception of medical cannabis. Throughout the paper it is clear there are other aspects being studies, however, in the discussion it states the overarching question for the study was whether consensus messaging influences public support for legalization of cannabis. This is not made clear at the start of the paper, and indeed there are several bits throughout the paper that set out how the study is being used to test different aspect of cannabis, and to see what impact the study design has consensus. This needs to be tidies up a bit and the focus of the paper clarified at the start/discussion. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes:** Anna Ross [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0260342.r001
Revision 1
21 Oct 2021 Author Response NOTE: I have also uploaded this information as a Response to Reviewers document Authors’ Responses to Reviewers Thank you for giving us the opportunity to revise our manuscript titled “Influences of Study Design on the Effectiveness of Consensus Messaging: The Case of Medical Cannabis” and resubmit for possible publication at PLOS One. Below, we include point-by-point responses to the comments provided by the reviewers. In addition to the changes that we describe below, we have also removed Figure 1. Although we created the images/messages and used open access stock photos, we were not able to secure the rights to publish the figure with the logo from the National Academies of Sciences, Engineering, and Medicine. Therefore, we cut the figure from the paper and direct readers to our project page on OSF.io to view the stimuli used in the study. Review Comments to the Author Reviewer #1: This was a well written and organized manuscript that explored the effectiveness of consensus and evidence-based messaging in viewpoints on medical cannabis. Overall, the authors found that a message reporting a high percentage of scientists agreeing is a more convincing than reporting existence of evidence. The researchers also found exposure effects related to research method. This is an important article as it can help media and researchers effectively get their message out and for researchers to choose the best method of testing method effectiveness. Thank you! REV: It was not clear in the introduction as to why medical cannabis was the topic of choice. The authors noted that "we are less concerned with increasing public support for medical cannabis than we are curious about the persuasiveness of different messaging strategies." which is fine, but this manuscript will attract readers that are concerned with this, therefore I think a bit more needs to be discussed on the topic. Our reason for emphasizing this is that we wanted to be clear that this is a manuscript that contributes to theory by examining the effectiveness of consensus messaging as a strategy for increasing public support for controversial policies (such as the legalization of cannabis). To address the reviewer’s concern, we have updated the “current study” paragraph by providing the reasons for which cannabis was chosen as the context for the paper. First, scientific consensus has been established for this issue, and second, the current policies in place do not align with the available scientific evidence. We also reworded the last line in this paragraph so that it does not downplay the context. It now states: “Using cannabis as an example, this research tests and challenges aspects of the Gateway Belief Model, which provides an explanation for how scientific consensus messaging may improve public support for policies related to publicly controversial science.” REV: My major concern with the manuscript was the control condition. In Figure 1 the two experimental conditions state the "cannabis is effective" while the control condition mentions CBD and the fact that is has no psychoactive effects. I am not sure if that is a good control condition to compare against the experimental conditions, since it is only mentioning CBD when the questions asked were about cannabis. What I would like to see is a discussion on how this issue may have affected the results. We added this to the limitations section of the paper. See this paragraph below: A third limitation is that our control condition may not have functioned as we had intended. We chose to make the control condition about ongoing research related to CBD because we wanted a control message that was tangentially related to cannabis but was not a consensus message and was not about medical uses of cannabis. We expected that there would be no change between pretest and posttest for this control condition (e.g., CBD). According to the FDA, marijuana is different from CBD (FDA, 2020). CBD is one compound in the cannabis plant, is not psychoactive (c.f., tetrahydrocannabinol, or THC), and is marketed in an array of health and wellness products in places where cannabis remains illegal (FDA, 2020). Most participants (87%) who were randomly assigned the control condition message about CBD understood that this message was NOT stating that scientific consensus exists, and they chose the option that indicated that research on the effectiveness of cannabis is still ongoing. Although the control condition message mentioned that “researchers are still investigating…” the topic of the investigation was CBD, and the participants may not have made a distinction between the two. In retrospect, we should have included a response option that specifically mentioned CBD and not cannabis. Importantly, though, we would still expect the message to work in a similar way as if it were clearly not about cannabis. Since participants generally seemed to understand that the message meant no consensus exists (because research is still ongoing), we would not expect change between pretest and posttest on our outcome variables. We would have only expected negative change for the control condition if the message had stated that there is scientific consensus that cannabis is NOT effective or if there was pretest sensitization. We have no reason to believe our results were due to the former as the manipulation check item suggests participants understood the purpose of the message. Thus, we do not believe our results were negatively affected by this potential issue. REV: Minor things REV: The resolution for the figure was too poor to read them comfortably. We converted our original images from pdf to tiff (300 dpi) as requested by PLOS One’s image submission guidelines. We agree that the images as they appear in the submission pdf are blurry. However, if you click the “Click here to access/download; Figure…” the downloaded versions of the figures are much clearer. REV: In the conclusion, please add a statement on study design your findings. We changed the first paragraph of the conclusions section to the following: This study provides more evidence that study design decisions influence the extent to which exposure to a consensus message influences public perceptions and indirectly influences policy support (as posed by the Gateway Belief Model). One such decision is the way in which consensus messages are described; our study adds to the literature suggesting that the descriptive norm/authority appeal strategy is more persuasive than describing the existence of substantial evidence. However, as Landrum and Slater discussed (2020), there are philosophical issues with treating the descriptive norm/authority appeal strategy as a “consensus” message as well as practical issues (e.g., there is not always an accurate measurement of the proportion of agreeing scientists). Reviewer #2: Development of concept using different messaging strategies is very interesting, well explained and referenced. Excellent range of testing used, interesting and statistically relevant outcome reported. Some very interesting findings in regard to a shift in perception post consensus messaging, and this would benefit from further exploration in other research papers. This paper has implications beyond the cannabis field and is an excellent contribution to the topic. Thank you! REV: One point that needs work on: REV: Overarching research question at the start of the paper is whether consensus message or evidence messaging influenced the perception of medical cannabis. Throughout the paper it is clear there are other aspects being studies, however, in the discussion it states the overarching question for the study was whether consensus messaging influences public support for legalization of cannabis. This is not made clear at the start of the paper, and indeed there are several bits throughout the paper that set out how the study is being used to test different aspect of cannabis, and to see what impact the study design has consensus. This needs to be tidies up a bit and the focus of the paper clarified at the start/discussion. Thank you for this comment. First, we changed the first line of the discussion to state “This study aimed to contribute to our understanding of the efficacy of consensus messaging by examining how researchers’ decisions about study design might influence study results, using medicinal cannabis as the context.” Furthermore, we realize that we were using “acceptance” and/or “support” generally to account for each of the different types of outcome variables. We have reworded the “current study” section of the paper to more specifically state that the goal of scientific messaging strategies has been to increase public support for policies (like legalization). We believe that this connects better to the beginning of the introduction which specifies that the Gateway Belief Model aims to explain how communicating about scientific consensus may indirectly influence change in people’s support for policies (line 43). Attachments Attachment Submitted filename: RESPONSE TO REVIEWERS.docx https://doi.org/10.1371/journal.pone.0260342.r002
9 Nov 2021 Decision Letter - Lucy J Troup, Editor Influences of Study Design on the Effectiveness of Consensus Messaging: The Case of Medicinal Cannabis PONE-D-21-22069R1 Dear Dr. Landrum, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Lucy J Troup, Ph.D Academic Editor PLOS ONE Additional Editor Comments (optional): Thank you so much for working with the reviewers comments to improve the manuscript. This is an important topic in light of many policy changes occurring around the use of cannabis as medicine. Reviewers' comments: https://doi.org/10.1371/journal.pone.0260342.r003
Formally Accepted
15 Nov 2021 Acceptance Letter - Lucy J Troup, Editor PONE-D-21-22069R1 Influences of Study Design on the Effectiveness of Consensus Messaging: The Case of Medicinal Cannabis Dear Dr. Landrum: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Lucy J Troup Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0260342.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .