PONE-D-20-37198

The Assessment of Physical Risk Taking: Construct Validation of a New Behavioral Measure

PLOS ONE

Dear Dr. Smith,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

I have received two reviews of the manuscript entitled “The Assessment of Physical Risk Taking: Construct Validation of a New Behavioral Measure” that you recently submitted to PLOS One. I am very glad to have received comments and evaluations of your manuscript from two researchers who are knowledgeable and experts in the area of decision-making and risk-taking. As you will see, the reviewers have offered many detailed points and constructive suggestions on how to improve the manuscript.

I have read the manuscript prior to receiving these reviews in order to have an independent perspective and then again with the reviews in hand. I found myself agreeing with the reviewers in almost every point and recommendation they raised. All of us agree that the work tackles an interesting topic. The paper is also well-written and structured. It is also very important that you've made available all data and relevant files .

However, at the same time the reviewers have raised concerns that prevents me from accepting the paper in its current form. I share all of these concerns; However, I believe that you can address these concerns in a revision, so I would like to encourage you to submit a revision. The main points relate to the main objective of the current work (which is rather unclear or unspecified) and how it contributes (and extends) existing research and literature in the area of risk assessment and preferences. In addition, there are some open questions about certain methodological features of the APRT and the data analyses that support the main conclusions.

What I expect is that you will take each of the concerns seriously and address these concerns in two ways. First, where possible, you should make changes in the manuscript to correct shortcomings that the reviewers have identified. (If there are comments that you do not find to be correct or apt, you should still consider that the incorrect perception is something that you might expect in other readers, so it would be helpful to take steps for the paper to anticipate such misperceptions and add clarifications in the text to prevent them.) The goal is to make your paper as accurate, scientifically responsible, interesting, and accessible to a wide range readers and researchers. 

The second way that I would like you to respond to the reviewers is to put a lot of effort into a careful cover letter that goes through the comments point by point, explaining how you addressed each comment and, if you disagree with a comment, why you disagree (and, if possible, how you altered the writing in anticipation that other readers might have similar concerns).

Please submit your revised manuscript by May 07 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Emmanouil Konstantinidis, Ph.D.

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include a caption for figure 1.

3. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This paper present a novel behavioral task aimed at assessing "physical risk taking". The authors claim that one of the task's main goals is to disentangle the relative influences of the i) probability and ii) size of the reward, as well as the iii) probability and iv) size of the loss that may result from taking a risk. I agree with the authors that improved behavioral tasks of risk taking are much needed. Moreover, the empirical study has been conducted carefully, and I appreciate that the materials and data are openly accessible. That said, I have several conceptual questions and comments, and overal the central goal and contribution of the paper remained somewhat unclear to me: Is it primarily to develop a novel measure for "physical" risk taking (if so: why exactly physical risk taking)? Is it to disentangle the potential drivers of risk taking as mentioned above (if so: why was this done in a "novel" domain such as "physical risk taking" and not using one of the more established domains)? Is it to study "multi-dimensional" risk problems (i.e., as opposed to the typical gambles, where the probability of a reward is often the inverse of the probability of a loss)? Is it to further examine self-regulation and impulsivity as drivers of risk taking (which seems to be implied by the hot / cold versions of the task)? Or is it about a combination of all of the above? Taken together, all in all I am not (yet) entirely convinced that this task lives up to the expectations, and several issues need to be addressed in order to better clarify the exposition of this research. I hope that my comments and questions below will be helpful to this end.

Introduction:

- The authors argue that "most definitions of risk focus exclusively on potential negative consequences". Although it is true that some disciplines adopt the view of risk taking as behavior with potentially harmful consequences, this general statement is not entirely correct. In economics (and economic psychology), for instance, risk is typically defined as "outcome variance" (see Schonberg et al., 2011) and hence "risk" also exist in gains-only settings.

- Relatedly, contrary to the portrayal in the introduction it is not the case that psychological theories of behavioral risk taking exclusively focus(ed) on the role of negative consequences. In fact, studies in the field of decision analysis (often using monetary gambles) have examined many risk-related phenomena purely in the gain domain (e.g., EU-theory). Consequently, the documentation of the "reflection effect" started to draw attention on differential effects of gains and losses, triggering the introduction of a reference point in prospect theory. To some extent, the relative importance of gains and losses has also been studied in the BART (which serves as an important starting point in the current paper), with respective models taking into account reward sensitivity, loss aversion, etc. (Wallsten et al., 2005).

- The authors write that "current behavioral task correlate with real-world risky behaviors broadly", but there is no reference to support this claim. In fact, accumulating empirical evidence indicates that behavioral tasks may lack external validity.

- Contrary to the authors' claim, many attempts have been made to decompose various tasks into basic behavioral components. As outlined above, this has been the case for the BART, but naturally also for the widespread monetary lotteries (i.e., the associated choice models often differentiate between the roles of gains / losses and the respective probabilities; cf. the fourfold pattern of risk). The same is true for the Iowa Gambling Task (cf. Yechiam, 2005).

- It is not quite true that the "injury probability condition is almost never considered in the BART" (p. 5; as a minor comment, it seemed a bit odd to me to use the term "injury" in the context of the BART, maybe consider using "loss" or a similar term). Specifically, the BART-models mentioned above (Wallsten et al., 2005) were specifically developed to account for (subjective) explosion probabilities (e.g., to compare to the objective explosion probabilities). Moreover, in more recent research, the impact of different [explosion] probability conditions was directly assessed with respective prompts and modeling (Schürmann et al., 2019; Steiner et al., 2020).

Behavioral task:

- The authors argue (p. 7) that previous tasks may be constrained as they consist of games without "a real-world equivalent". In line with this argument, it has been proposed in the literature that some tasks (including the BART) may be constrained in terms of their generalizability (see comment above) because they lack a representative design (Steiner et al., 2020). Thus, it would be important to introduce how / why the four settings implemented in the APRT are ecologically more relevant (e.g., encountering a tiger does not seem an overly realistic real-life experience).

- The authors claim that an advantage of the novel task is that a participant cannot simply infer the probability of a gain from the probability of a loss (or vice versa), as there are also "neutral" outcomes. Thus, can one argue that this is a three-outcome setting? If so, it would be interesting to relate that to the respective research (e.g., on monetary gambles) that has used setups other than the typical two-outcome gambles.

- Relatedly, I wondered what to make of the additional dimension of "health". It seemed to me as if the participants' had to deal with two non-fungible "currencies" in the task (i.e., maximize payoffs in terms of the points, or maximize health)? Can the authors elaborate further on this issue?

- Finally, I was also somewhat puzzled by way the information about the probabilities of success / failure was communicated to participants (the colored / striped borders). The authors argue that this procedure "protects the task from learning effects", but I could not quite follow this rationale. In fact, this creates an additional learning component exactly *because* participants have to remember (and recall) the meaning of this symbolic representation (which in itself could lead to noisier behavior). Second and under the assumption that participants could perfectly recall the meaning of the various borders, how / why would this be any different from directly depicting the probabilities?

Hypotheses / analyses:

- I found the theoretical motivation for the various hypotheses (p. 11) relatively thin. This is particularly an issue as the model implemented to test these hypotheses is very flexible (i.e., many terms and interactions; generally many exploratory analyses). Relatedly, although participants completed a number of trials, I wondered whether the various parameters are fully identified in this model. A simulation / model recovery analysis could be informative and boost the confidence in the robustness of the results.

Schonberg, T., Fox, C. R., & Poldrack, R. A. (2011). Mind the gap: Bridging economic and naturalistic risk-taking with cognitive neuroscience. Trends in Cognitive Sciences, 15(1), 11–19. https://doi.org/10.1016/j.tics.2010.10.002

Schürmann, O., Frey, R., & Pleskac, T. J. (2019). Mapping risk perceptions in dynamic risk-taking environments. Journal of Behavioral Decision Making, 32, 94–105. https://doi.org/10.1002/bdm.2098

Steiner, M. D., & Frey, R. (2020). Representative Design in Psychological Assessment: A Case Study Using the Balloon Analogue Risk Task (BART). https://doi.org/10.31234/osf.io/dg4ks

Wallsten, T. S., Pleskac, T. J., & Lejuez, C. W. (2005). Modeling behavior in a clinically diagnostic sequential risk-taking task. Psychological Review, 112(4), 862–880. https://doi.org/10.1037/0033-295X.112.4.862

Yechiam, E., Busemeyer, J. R., Stout, J. C., & Bechara, A. (2005). Using cognitive models to map relations between neuropsychological disorders and human decision-making deficits. Psychological Science, 16(12), 973–978. https://doi.org/10.1111/j.1467-9280.2005.01646.x

Reviewer #2: The manuscript “the assessment of physical risk taking: construct validation of a new behavioral measure” presents a new method of assessing physical risk taking which simultaneously varies probability and reward in more realistic contexts than existing behavioral measures. As an initial presentation, the paper is adequate in it’s rationale and uniqueness. The data presentation is fairly thorough and there is some initial evidence of construct validity.

Therefore as an initial presentation I believe the manuscript is adequate, however I believe the limitations and future directions should be more explicit on what is not in the manuscript (and given the scope of the paper probably shouldn’t be in this paper). First, construct validity is not a conclusive state but a process of accumulation that would need to continue in future studies. Second, criterion validity is absent and would be required before the measure could be used in any real-world applications. Third, a direct comparison between the APRT, BART, IGT, and other major behavioral risk-taking measures would be required to illustrate the practical advantages the APRT claims. Fourth, the issue of non-response mentioned in the limitations needs further study with larger and more diverse samples. And finally, given all of these points, it should be made clear that the APRT requires further evidence of validity and utility prior to any applied use.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-20-37198R1

The Assessment of Physical Risk Taking: Preliminary Construct Validation of a New Behavioral Measure

PLOS ONE

Dear Dr. Smith,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

I have now received one expert review of your manuscript; the reviewer agree that you present a much improved version of the manuscript. While most of the comments have been addressed in this revision, the reviewers have suggested a few areas that the manuscript can be improved (mostly minor comments). I also believe that the manuscript will benefit from a final revision of the submitted work which addresses the reviewer's comments.

Please submit your revised manuscript by Sep 24 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Emmanouil Konstantinidis, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have carefully responded to all of my previous comments and questions, and I believe that the revised paper now does a much better job in highlighting the contributions (and limitations) of the current study. I have a number of final suggestions that may help to further strengthen the exposition of this work.

1. On p. 4 the authors argue that previous tasks do not correlate strongly with real-life behaviors and hence, that there is a need for more complex behavioral tasks. Alike, on p. 46 the authors argue that maybe even "more complex simulations" are required (i.e., to capture additional dimensions relevant to risk taking). So this is actually rather a comment than a suggestion, but personally I do not quite agree with this conclusion. In my view many tasks are already too complicated, often making it difficult to identify the exact reasons why people do (vs. do not) take risks in these tasks (especially if only few trials are collected; see also comment 3 below). An alternative and in my view more promising approach consists of actually developing simpler tasks, which may capture specific risk-related dimensions more precisely. Again, this is my personal opinion and should not prevent publication of this paper; the respective debate needs to be conducted in the field.

2. The study on the novel task aims to achieve many things at once (i.e., multiple conditions and various outcomes are compared against multiple measures and scales), and ultimately the reader may wonder how robust the reported effects indeed are. I appreciate that the authors acknowledged the exploratory nature of their research in the revision and that they sharpened their predictions where applicable. In my view, however, it could be helpful to make the exploratory nature even more explicit, in order not to raise any wrong expectations. For example, adding something like "An exploratory investigation of..." to the title and / or abstract may help setting expectations straight.

3. With comments #1 and #2 in mind, I would find it essential that future research clarifies the identifiability and reliability of the parameters estimated in this task. Again, many things are being manipulated and measured here – but are the various dimensions indeed clearly separable, and is the number of trials sufficient to identify reliable response patterns? As outlined in my previous review, a model recovery analysis could be very instrumental to this end. Briefly, the idea is to implement various data-generating processes (e.g. simulate groups of people who take risks for different reasons in different conditions), to then estimate the models and evaluate whether the implemented effects can be recovered (or how strong these effects would need to be in order to be recovered reliably). The authors responded that they are not confident implementing such an analysis - fair enough, but it would be very helpful to have at least an explicit discussion of the option to run a parameter recovery study in the future, to thus clarify the reliability of the extracted parameters / effects.

4. Finally and also with expectation management in mind, I wondered whether it would not be more accurate / informative to use "simulation of" instead of "assessment of" physical risk taking in the name of the task. The latter implies that some concrete real-life behaviors are assessed, which is naturally not quite the case as the authors also acknowledge in the paper.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

The Assessment of Physical Risk Taking: Preliminary Construct Validation of a New Behavioral Measure

PONE-D-20-37198R2

Dear Dr. Smith,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Emmanouil Konstantinidis, Ph.D.

Academic Editor

PLOS ONE