A randomised experiment of health, cost and social norm message frames to encourage acceptance of swaps in a simulation online supermarket

Amanda Bunten; Lucy Porter; Jet G. Sanders; Anna Sallis; Sarah Payne Riches; Paul Van Schaik; Marta González-Iraizoz; Tim Chadborn; Suzanna Forwood

doi:10.1371/journal.pone.0246455

Peer Review History

Original SubmissionOctober 12, 2020
4 Dec 2020 Decision Letter - Zhifeng Gao, Editor PONE-D-20-32025 A randomised controlled experiment of health, cost and social norm message frames to encourage acceptance of food swaps in a virtual supermarket PLOS ONE Dear Dr. Porter, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Both reviewers think the paper is interesting. However, they also point out the two major problems. 1. Inadequate sample size. 2. The lack of control in the experiment. I hope you can sufficiently address these comments in your revision. Please also reply to the reviewers' comments item by item following the instruction below. Please submit your revised manuscript by Jan 18 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols We look forward to receiving your revised manuscript. Kind regards, Zhifeng Gao Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please consider modifying your title to ensure that it is specific, descriptive, concise, and comprehensible to readers outside the field, by specifying that the study used a simulation approach. This should also be clear in the Abstract. 3.In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability. Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized. Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access. We will update your Data Availability statement to reflect the information you provide in your cover letter. 4. Please ensure that you refer to Figure 1 in your text as, if accepted, production will need this reference to link the reader to the figure. 5. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Partly Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: I enjoyed reading this very interesting article on framing effect and food swap. The article addresses a very practical issue on how to “nudge” people for a last-minute healthy alternative. The manuscript is comprehensive and well written. As the authors pointed out in the introduction and discussion, this is a hypothetical experiment conducted online. The authors attempted to compare cost frame and social norm frame to the health frame. My major concern is that the experiment does not have a control group. What is inherent in a health frame is unknown, therefore the difference between a health frame and a cost frame is implicit. The same goes for the social normal frame. Is being healthy considered as part of the social norm? without a true control group that has no framing at all, it is hard to interpret the findings. This is perhaps why the authors struggled with explaining the null effect - because it is not really a null effect, it is a health effect. Since the experiment was conducted online, the authors should be able to collect another control group, bearing that the new sample is random and independent from the three framing groups. If for some reason, the authors are unable to collect a control group, analysis plan should be adjusted to use descriptive statistics and statistical tests to analyze the difference among the three groups, rather than pooling all data in a regression framework. Minor The motivation for presenting models 1-3 is unclear. Unless you are using a different dependent variable in each model, you should only present the preferred model. In table 2, there seem to be some difference in education between the three conditions. A statistical test should be conducted on the differences. Thank you for the opportunity to learn about your work. Reviewer #2: This is the second time I am reviewing this paper. I am pleased that the authors have taken most of my previous comments into consideration. I hope the comments below can help strengthen the manuscript. The main limitation of the manuscript are that the proposed sample size was not reached and therefore we do not know if there was actually evidence of no effect or no evidence of effect. However, it is an otherwise well-conducted and well-reported study that can add to the literature of how to assess potential new interventions to promote healthier choices in online supermarkets. Such studies are urgently needed, particularly given the change in shopping practices and dietary habits due to COVID-19. Abstract Reword kj/kcal to kJ or kcal, as it may cause confusion. The presentation of the results in the abstract does not align with the randomised design. For example, you need to first present the results by trial group and then specify that when all groups were combined in a pre-post design there was an indication of effect. “Predictor” has very specific implications, rewording this to moderator throughout is suggested. Also specifying that this was exploratory. Background Well written, logic flow, and makes the case for the need for this study. Line 54: Suggest you add some non-UK data too to appeal to an international audience. Methods Line 109: Delete “was”. Line 114: Did you take into account dropouts in the sample size calculation? Line 123: More detail is needed re the method and process of randomisation and allocation concealment. See CONSORT guidance. Line 151: Were the two swaps appeared simultaneously (ie. Next to each other) or did the second only appear after the first was rejected. Please add information on whether you considered other characteristics (e.g. taste proximity, brand) between offered product to the originally selected, as a same-brand low-kcal cheddar cheese might be much more acceptable than a random one. It would be extremely useful for the readers to grasp the intervention if Table 1 was replaced by an example snapshot of the online supermarket while swaps were offered. Line 190: “was planned to” might imply that this was an original plan but what happened in reality differed. Worth clarifying. Lines 194, 224 and throughout: I assume authors mean sex rather than gender? Line 210: Suggest reword analysis (ii) to “effect of swap frame on energy reduction” Line 290: Reword “was” to “would have been” Worth adding p-values for comparisons in Figure 1. Discussion Well-balanced. I would add in the cost paragraph that potentially the lack of minimal cost cut-off might have contributed to no effect. It is possible that if something was simply £0.01 cheaper, people wouldn’t want to sacrifice their original choice for such a small benefit. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes:** Dimitrios Koutoukidis [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0246455.r001
Revision 1
13 Jan 2021 Author Response (Note: we have also uploaded this response as a file for ease of reading re: distinguishing between reviewer advice and our response.) Dear Dr. Gao, Thank you very much for giving us the opportunity to respond to the reviewers’ comments. We have responded to each of them below and hope that the amendments we have made will meet your requirements. Response to Reviewer #1 I enjoyed reading this very interesting article on framing effect and food swap. The article addresses a very practical issue on how to “nudge” people for a last-minute healthy alternative. The manuscript is comprehensive and well written. As the authors pointed out in the introduction and discussion, this is a hypothetical experiment conducted online. The authors attempted to compare cost frame and social norm frame to the health frame. My major concern is that the experiment does not have a control group. What is inherent in a health frame is unknown, therefore the difference between a health frame and a cost frame is implicit. The same goes for the social normal frame. Is being healthy considered as part of the social norm? without a true control group that has no framing at all, it is hard to interpret the findings. This is perhaps why the authors struggled with explaining the null effect - because it is not really a null effect, it is a health effect. Since the experiment was conducted online, the authors should be able to collect another control group, bearing that the new sample is random and independent from the three framing groups. If for some reason, the authors are unable to collect a control group, analysis plan should be adjusted to use descriptive statistics and statistical tests to analyze the difference among the three groups, rather than pooling all data in a regression framework. Thank you very much for your comments, we are glad that you found the manuscript comprehensive and well-written. In response to the concern regarding the Control group, we agree that the Active Control group (i.e., the health frame) used in this study is conceptually different from the idea of a “no frame” Control group that offers swaps without an accompanying message. However, the research question of focus for this study was whether framing swaps in terms of cost or social norms was a more effective strategy than framing in terms of health benefits. Given this, using the health frame group as a comparison group is the most appropriate strategy for answering this question. Nevertheless, your comments encouraged us to think about this issue more broadly and we have made a couple of amendments to reflect this, which we hope you will approve of. Firstly, we have reconceptualised the description of the study as a “randomised experiment” (rather than a randomised controlled experiment) to reflect the fact that the Control group in this study still involved exposure to an intervention. Secondly, after reflecting on your comment about what is inherent in a health/social norm/cost frame, we arrived at the conclusion that in this experiment, even a no-frame condition could be considered as inherently health-related. This is because the products offered in exchange always had a lower energy content, and while this may not have always been apparent (e.g., when one type of “full-fat/full-sugar” sweet was suggested as a swap for another full-fat/sugar sweet), in a number of instances, participants would have been presented with an option that was evidently lower energy (e.g., due to branding indicating “light” products). This would have been the same across conditions, regardless of whether the message frame drew attention to the energy content of the product. We have therefore added this to the discussion as a limitation. The motivation for presenting models 1-3 is unclear. Unless you are using a different dependent variable in each model, you should only present the preferred model. Thank you for your comment – we have reflected on this and have decided to remove model 3 as this was an exploratory question and was not specified in our pre-registration. Model 2 (i.e., that adjusted for demographic variables) is considered our preferred model, however we have also included the unadjusted model of intervention group alone (i.e., Model 1) as standard. The models are now referred to as the “adjusted” and “unadjusted” models, which we hope will aid understanding. In table 2, there seem to be some difference in education between the three conditions. A statistical test should be conducted on the differences. Thank you for the opportunity to learn about your work. Thank you for your comment – we had originally conducted these tests for all of our baseline characteristics, however an earlier reviewer instructed us to remove these, citing the CONSORT guidelines and the paper by de Boer, Waterlander, Kuijper, Steenhuis and Twisk (2015) which makes the case for not doing so. Given these guidelines, we have not included them in the manuscript but our earlier test found that the difference in education was not significant across conditions (p = .774). Thank you very much for taking the time to review our work. We hope that the amendments made are to your satisfaction. Response to Reviewer #2 This is the second time I am reviewing this paper. I am pleased that the authors have taken most of my previous comments into consideration. I hope the comments below can help strengthen the manuscript. The main limitation of the manuscript are that the proposed sample size was not reached and therefore we do not know if there was actually evidence of no effect or no evidence of effect. However, it is an otherwise well-conducted and well-reported study that can add to the literature of how to assess potential new interventions to promote healthier choices in online supermarkets. Such studies are urgently needed, particularly given the change in shopping practices and dietary habits due to COVID-19. Thank you for your comments, and for taking the time to review our paper for the second time. We agree that the reduced sample size is a limitation for the manuscript, and we have now emphasised this more in our discussion. Your comment encouraged us to reflect further on this point and we believe that lowered power cannot entirely explain our results as the mean trend (while non-significant) showed a pattern in the opposite direction to the one we expected. As a result, we believe that other factors (such as the fact that all participants were asked to stick to a budget, or the fact that all participants could see price information) may be more important for the null results observed here. We have emphasised this point too. We hope that the methodological learnings from this paper can be of use for others developing and evaluating interventions in the field, and believe that our paper makes a useful contribution in this regard. Abstract Reword kj/kcal to kJ or kcal, as it may cause confusion. Thank you, we have reworded this. The presentation of the results in the abstract does not align with the randomised design. For example, you need to first present the results by trial group and then specify that when all groups were combined in a pre-post design there was an indication of effect. Thank you, we have now reordered the presentation of results. “Predictor” has very specific implications, rewording this to moderator throughout is suggested. Also specifying that this was exploratory. Thank you for your suggestion, we don’t believe that moderator is the right term for this relationship as (for example, in this case) the age � swap acceptance analysis was a main effect rather than an interaction. We have reworded to describe “associations” instead. Background Well written, logic flow, and makes the case for the need for this study. Line 54: Suggest you add some non-UK data too to appeal to an international audience. Thank you, we have added some non-UK data to describe global shopping habits, particularly contextualised within the coronavirus pandemic. Methods Line 109: Delete “was”. This has been done. Line 114: Did you take into account dropouts in the sample size calculation? No, dropouts were not accounted for. This is now clearer in the manuscript. Line 123: More detail is needed re the method and process of randomisation and allocation concealment. See CONSORT guidance. Thank you for highlighting this, we have now provided more information on the randomisation allocation ratio and allocation concealment. Line 151: Were the two swaps appeared simultaneously (ie. Next to each other) or did the second only appear after the first was rejected. Both appeared simultaneously. We have now amended this to make it clearer. Please add information on whether you considered other characteristics (e.g. taste proximity, brand) between offered product to the originally selected, as a same-brand low-kcal cheddar cheese might be much more acceptable than a random one. Thank you, more information has been added to the manuscript. To summarise in brief, the algorithm did not consider characteristics such as taste proximity and brand. Alternative products were selected from within the same shelf. Unfortunately, this means that some swaps were not “in kind” (e.g., a pot of salsa being suggested as a swap for crisps). We have added this as a limitation to the discussion. It would be extremely useful for the readers to grasp the intervention if Table 1 was replaced by an example snapshot of the online supermarket while swaps were offered. Thank you – we do not have screenshots of the intervention, however we have created mock-ups for readers to see. Line 190: “was planned to” might imply that this was an original plan but what happened in reality differed. Worth clarifying. Thank you, we have reworded. Lines 194, 224 and throughout: I assume authors mean sex rather than gender? The wording of the questionnaire that participants saw was gender, and so this term is retained. Line 210: Suggest reword analysis (ii) to “effect of swap frame on energy reduction” Line 290: Reword “was” to “would have been” Worth adding p-values for comparisons in Figure 1. Discussion Well-balanced. I would add in the cost paragraph that potentially the lack of minimal cost cut-off might have contributed to no effect. It is possible that if something was simply £0.01 cheaper, people wouldn’t want to sacrifice their original choice for such a small benefit. Thank you for the suggestions, we have amended as advised. Thank you again for your time and contributions to this manuscript on its second round, we hope that the amendments have been made to your satisfaction. https://doi.org/10.1371/journal.pone.0246455.r002
20 Jan 2021 Decision Letter - Zhifeng Gao, Editor A randomised experiment of health, cost and social norm message frames to encourage acceptance of swaps in a simulation online supermarket PONE-D-20-32025R1 Dear Dr. Porter, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. However, reviewers and I both noticed some typos, please carefully check your final manuscript to make corrections. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Zhifeng Gao Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: This revision has significantly improved based on my comments. There are some typos in the manuscript. Please check carefully based on the Journal's format guide. Reviewer #2: Thank you for the revised manuscript, my comments have been addressed. I have nothing further to add. ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes:** Dimitrios Koutoukidis https://doi.org/10.1371/journal.pone.0246455.r003
Formally Accepted
26 Jan 2021 Acceptance Letter - Zhifeng Gao, Editor PONE-D-20-32025R1 A randomised experiment of health, cost and social norm message frames to encourage acceptance of swaps in a simulation online supermarket Dear Dr. Porter: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Zhifeng Gao Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0246455.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .