PONE-D-21-06378

Privacy Nudges for Disclosure of Personal Information: A Systematic Literature Review and Meta-Analysis

PLOS ONE

Dear Dr. Ioannou,

Thank you for submitting your manuscript to PLOS ONE. The paper has been carefully considered and appreciated, but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Particular care during the revision should be devoted to thoroughly address reviewers’ methodological comments. Substantial improvements are expected and required for a positive evaluation. For each of the points raised by the reviewers, a specific comment is required.

Specifically, Reviewer 1 raises a major concern with regard to the choice of almost completely rely on  Acquisti et al.’s (2017) taxonomy of nudges. Reviewer 2 comments on some limitations of that taxonomy and suggests extensions. This seems a particularly relevant point calling for a detailed examination.

Reviewer 2 presented several observations that require careful consideration in order to improve the paper.

Both reviewers commented on several imprecisions in quoting and in citations that should be fixed in the next version.

Please submit your revised manuscript by May 23 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Marco Cremonini, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This paper presents a meta-analysis of literature on digital nudging. It is well written, coherent, clear, and well structured. It steers clear of the question regarding the overall effectiveness of digital nudging, as this would require knowing about the studies with a null result that were never published. In other words, the paper shows an awareness of publication bias in experimental studies. The paper therefore limits itself to an analysis the corpus of studies that it collected using clear and consistent criteria. I enjoyed reading it and I believe it can make an important contribution to the literature on digital nudging.

General comments

I have one major concern. The authors rely on Acquisti et al.’s (2017) taxonomy of nudges, and group their studies according to the categories of information, presentation, defaults, and incentives. They neglect reversibility (error resiliency) and timing, which they should acknowledge and possibly explain. Peer and Acquisti, 2016, for example, included in the meta-analysis, would seem to fit under reversibility. However, the main problem lies with the category ‘incentives’.

The whole point of nudging is to go beyond ‘traditional’ policy approaches to influencing behaviour, like bans or taxes. To influence behaviour by offering an incentive is not behavioural economics at all, it is simply economics. It is the most basic, traditional economic approach to influencing behaviour. This is why the demand curve slopes downward and the supply curve slopes upward: because price matters. A monetary incentive, therefore, should not be considered a nudge. The authors acknowledge this when they quote Thaler and Sunstein’s (2008) definition of a nudge on page 3: a nudge is ‘any aspect of the choice architecture that alters people’s behavior in a predictable way without forbidding any options or significantly changing their economic incentives.’

I realise the authors are relying on an existing taxonomy, namely Acquisti et al.’s. But it seems to me, from a cursory glance, that Acquisti et al. had a more nuanced view of incentives. In some cases, rewards were non-monetary. In others, they intended to offset existing biases, such as hyperbolic discounting. Still, in other cases, biases such as loss aversion were leveraged to shape the perception of rewards and punishments. I would invite the authors to reflect on this problem. If the use of the incentive category is justified (à la Acquisti et al.), they should explain this. If it is not, the incentive category and the four papers within it should be excluded.

Specific comments

- The paper is a bit sloppy when quoting references. For one, it seems to have been written with a different referencing system in mind (Harvard or APA), which sometimes leads to inelegant phrasing, e.g. ‘[8] sought to nudge users…’ Acquisti et al. (2017) appears in the text without its reference number. On pages 21 and 22, references 38 and 65 are mentioned twice. The paper would benefit from a proofreading throughout to address these issues.

- On page 11, the paper claims ‘the investigation of privacy nudges has grown significantly during the last decade, since 2013’. Strictly speaking, given the time lag in scientific publications, the growth in investigation probably began in 2011 or 2012.

- In Section 3.3.4, first par: ‘Research has shown that the higher the amount of the monetary incentive, the higher the percentage of people who shared their information [81].’ Monetary incentive for what? For disclosing or not disclosing? Common sense would dictate that it is for disclosing, but this should be clarified.

- On page 31, the paper claims ‘results of the meta-analysis show that in absolute terms incentives had the strongest effect on users’ information disclosure’, which is not surprising, as incentives work on a different level (as I noted above). I wonder if this strong effect pushes the overall effect of the studies over a certain threshold. In other words, what would be the overall effect of the studies if we excluded incentives? Given that incentives are not really nudges, it would be good to know this. Would conclusion no (2) ‘a meta-analysis of an adequate papers revealed that nudging strategies can effectively influence information disclosure’ still hold?

- On page 32, the paper claims that the number of studies in the ‘incentive’ category were five, but elsewhere it says they were four. I guess it means four studies, but five effects. Could you please clarify?

Reviewer #2: This is an interesing study. You need to some work on the paper if you want to get it published

(1) People do not make imperfect decisions - to them, the decisions are perfect. They make unwise decisions

(2) You have a habit of not using the author name. eg [1] says .... Respect the author enough to name them

(3) When a nudge encourages unwise disclosure, it is no longer a nudge which must always be for the good of the nudgee. What it becomes is SLUDGE

(4) You are conflating privacy and security. Malware is a security problem. Disclosure is a privacy issue. If malware makes data inaccessible, that is an availabiity issue (from the CIA principles) and thus security.

(5) when quoting, provide page numbers

(6) Personal infomation should include mention of telephone numbers and email addresses

(7) sec 3.4 referencing problem in first line

(8) p26 - last para - what does 84 refer to?

(9) p31 - again, when a nudge is not doing good to the nudgee, it is sludge

(10) provide a reference for incentives. BTW - nudges, per definition, exclude incentives (rewards) or sanctions. Strictly speaking, anything using an incentive is not a nudge.

(11) The conclusions and implications section is FAR too long. Call this reflection and tighten it up. The section rambles too much

(12) add a short and concise conclusion

(13) at the end you say that more work needs to be done. This is standard in future work sections but disatisfactory. Say how - you know the literature so say exactly how this should be achieved.

(14) incomplete refs: 2, 13,, 15, 24, 35, 29, 48, 95 (is xx correct?), 103 - what is CNBC?, 105 (repeats at end), 46 repeats at end

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Karen Renaud

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-21-06378R1

Privacy Nudges for Disclosure of Personal Information: A Systematic Literature Review and Meta-Analysis

PLOS ONE

Dear Dr. Ioannou,

Thank you for submitting your manuscript to PLOS ONE. Reviewers have carefully read your manuscript and overall they agree that it clearly improved from the original submission. A remaining open issue, however, still requires further consideration. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

In particular, Reviewer1 has commented on a clearly relevant issue concerning incentives. The manuscript could benefit from the inclusion of a clear distinction between nudges and incentives, a discussion on the inclusion or exclusion of incentives from models, and in general a clear presentation of the issue and of relevant approaches regarding this key aspect. Reviewer1's suggestion is to give to this discussion more space and more evidence. Authors are invited to further consider this aspect and submit a revised manuscript or provide a detailed rebuttal of the suggestion.

Please submit your revised manuscript by Aug 16 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Marco Cremonini, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: (No Response)

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for taking my comments into account. Regarding incentives in particular, I am reassured that running the analysis without incentives does not change the main outcome of the paper, which was a big worry. However, I am disappointed that you did not give this issue greater importance in the paper. It is reduced to a footnote and to an analysis in the Supplementary Material. You did not take up my invitation to reflect more on the question of incentives. Just hiding behind Acquisti et al.'s taxonomy as a justification is not enough - in my opinion - to advance scientific knowledge on this issue and move the debate forward. At the very least, you have a glaring contradiction in your paper, which has "Nudges" in its title, cites Thaler and Sunstein's definition that an incentive is *not* a nudge, and then happily embraces Acquisti et al.'s taxonomy without questioning whether the inclusion of incentives is appropriate in this context. I would have expected at least a paragraph where you discuss this issue. You could argue that perhaps it depends on the behaviour being observed. Or you could go into the detail of how Acquisti et al. apply the category of incentives in their study (as I mentioned in my review). Perhaps in the context of privacy an incentive can and should be considered a nudge, in apparent contradiction to Thaler and Sunstein. But the reader would like to know why. I would like to insist on such a reflection on your part. I think it is important, and if we start overlooking these important issues, lying at the core of the behavioural turn in policy-making, the value of the approach will be undermined and it will all start to go downhill from there. Please provide at least a paragraph, possibly in Section 3.3.4, where you address this so that the reader who is interested in nudges and privacy can better understand if - in the context of privacy and following the definition of Thaler and Sunstein - an incentive can or cannot be considered a nudge.

Reviewer #2: I'm happy with the revisions the authors made. They have been responsive to my comments and the paper is much improved

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Privacy Nudges for Disclosure of Personal Information: A Systematic Literature Review and Meta-Analysis

PONE-D-21-06378R2

Dear Dr. Ioannou,

The revised paper has successfully addressed all reviewers' comments, therefore we’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Marco Cremonini, Ph.D.

University of Milan

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments: