PONE-D-25-01433Development and Validation of the Self-Awareness of Ego-Threatening Biases QuestionnairePLOS ONE

Dear Dr. Baumann,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

For your revision, we kindly ask that you pay special attention to the following points, among other comments raised by the reviewers:

• Separate the Introduction and Literature Review sections.
• Add a subsection to clarify the hypotheses.
• Reconsider the validity of the simplified one-factor model used in Study 1.
• Temper the claims in Study 2, as they rely on the preliminary findings of Study 1.
• Improve the reproducibility of the R code, including clearer documentation and environment specifications.

Please submit your revised manuscript by May 31 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Hidenori Komatsu

Academic Editor

PLOS ONE

Journal requirements: 1. When submitting your revision, we need you to address these additional requirements. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information. 3. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please move it to the Methods section and delete it from any other section. Please ensure that your ethics statement is included in your manuscript, as the ethics statement entered into the online submission form will not be published alongside your manuscript. 4. We note that this data set consists of interview transcripts. Can you please confirm that all participants gave consent for interview transcript to be published? If they DID provide consent for these transcripts to be published, please also confirm that the transcripts do not contain any potentially identifying information (or let us know if the participants consented to having their personal details published and made publicly available). We consider the following details to be identifying information:- Names, nicknames, and initials- Age more specific than round numbers- GPS coordinates, physical addresses, IP addresses, email addresses- Information in small sample sizes (e.g. 40 students from X class in X year at X university)- Specific dates (e.g. visit dates, interview dates)- ID numbers Or, if the participants DID NOT provide consent for these transcripts to be published:- Provide a de-identified version of the data or excerpts of interview responses- Provide information regarding how these transcripts can be accessed by researchers who meet the criteria for access to confidential data, including:a) the grounds for restrictionb) the name of the ethics committee, Institutional Review Board, or third-party organization that is imposing sharing restrictions on the datac) a non-author, institutional point of contact that is able to field data access queries, in the interest of maintaining long-term data accessibility.d) Any relevant data set names, URLs, DOIs, etc. that an independent researcher would need in order to request your minimal data set. For further information on sharing data that contains sensitive participant information, please see: https://journals.plos.org/plosone/s/data-availability#loc-human-research-participant-data-and-other-sensitive-data If there are ethical, legal, or third-party restrictions upon your dataset, you must provide all of the following details (https://journals.plos.org/plosone/s/data-availability#loc-acceptable-data-access-restrictions):a) A complete description of the datasetb) The nature of the restrictions upon the data (ethical, legal, or owned by a third party) and the reasoning behind themc) The full name of the body imposing the restrictions upon your dataset (ethics committee, institution, data access committee, etc)d) If the data are owned by a third party, confirmation of whether the authors received any special privileges in accessing the data that other researchers would not havee) Direct, non-author contact information (preferably email) for the body imposing the restrictions upon the data, to which data access requests can be sent 5. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1:  Dear Authors,

Congratulations on preparing an excellent manuscript. I thoroughly enjoyed reading your work and believe it offers meaningful and insightful contributions to the field. The newly developed questionnaire will undoubtedly open valuable avenues for future research. Both studies presented were thoughtfully designed, with appropriate data analysis methods. Additionally, your discussions related to the results are insightful and relatively comprehensive. However, I have several suggestions to further strengthen the literature review section.

First, the introduction and literature review sections currently lack clear separation and detailed elaboration. Although the research background was briefly introduced at the outset, gaps identified clearly, and key concepts such as biases and self-awareness introduced effectively, a clearer and more structured distinction between these sections would significantly improve readability and coherence.

Second, your literature review could benefit from deeper exploration, particularly regarding the connection between self-awareness and metacognition. Given the importance of the metacognition concept to your study, expanding the discussion on how self-awareness specifically relates to metacognition would greatly enhance the relevance and depth of your theoretical framing.

Furthermore, while you have successfully introduced the broader research context, highlighted existing challenges, and summarized past literature relevant to your hypotheses, the literature review would be even stronger if you included a critical analysis of past studies. Please consider explicitly discussing the strengths, limitations, and key lessons derived from previous research. This approach would better position your study within existing literature and underscore its contributions.

Lastly, I recommend dedicating a specific subsection to clearly outline your hypotheses. It would be particularly beneficial if you explicitly connect these hypotheses to the research gaps you aim to address, emphasizing how your research introduces novelty. Ideally, this subsection should serve as a concise summary highlighting the significance and rationale for conducting your research, clearly linking research gaps, hypotheses, and anticipated contributions.

Wishing you all the best with your publication.

Kind regards,

Reviewer

Reviewer #2:  Acknowledgments:

I would like to acknowledge that my expertise lies primarily in statistical modeling, psychometric validation, and methodological rigor rather than psychological theory or clinical interpretation. My review thus focuses explicitly on the statistical and methodological choices, rigor, transparency, and reproducibility of the analyses conducted. Interpretations of psychological constructs or theoretical frameworks should also be evaluated by reviewers whose expertise aligns more closely with these aspects.

Methodology: Overall Thoughts and Recommendations

The manuscript is partly technically sound: statistical analyses performed are correct given their methodological choices, but foundational methodological decisions (factor structure choice, transparency, reproducibility) require significant improvement.

1. The authors’ findings linking self-awareness of ego-threatening biases (SAETBQ) and moral disengagement are intriguing but potentially problematic due to methodological choices. Specifically, adopting a simplified one-factor solution despite clear statistical support for a four-factor structure risks oversimplifying complex psychological relationships, thereby influencing both the interpretation and practical implications of the results. Given the ethically sensitive nature of the conclusions drawn, the authors should explicitly acknowledge how their decision to use a single-factor model limits interpretive clarity, nuance, and generalizability. This limitation underscores the need for further investigation and caution before drawing definitive or potentially stigmatizing conclusions.

2. The authors’ claim that “the findings of Study 1 show that we can reliably and validly assess self-awareness of ego-threatening biases with a 12-item questionnaire (SAETBQ)” appears overstated, given significant unresolved methodological concerns. The claimed reliability and validity depend heavily on a simplified one-factor model selected despite robust statistical evidence favoring a more nuanced four-factor solution. Given that Study 2 relies entirely on this measure, the authors should explicitly temper this claim by clearly acknowledging the preliminary nature and methodological limitations of Study 1’s findings.

Computational Methods and Data Validation:

The authors have commendably made their dataset publicly available, significantly supporting transparency. However, the supplementary R code intended to enable independent reproduction of their analyses lacks critical details necessary for genuine reproducibility. Specifically:

1. Insufficient documentation and explanatory comments, making it difficult to follow the logic or interpret intermediate outputs clearly.

2. No specification of the R version, package dependencies, or computational environment (e.g., no use of standard reproducibility tools such as sessionInfo() or renv).

3. Absence of a clearly structured workflow or explicit instructions for execution, hindering straightforward reproduction attempts.

For the sake of transparency, rigor, and reproducibility—fundamental principles in statistical modeling and psychometric validation—these issues must be explicitly addressed. The authors are encouraged to substantially improve code documentation, explicitly provide computational environment details, and clearly structure the analytical workflow.

References:

A brief review of the manuscript’s references reveals a notable absence of citations to foundational methodological literature on Exploratory Factor Analysis (EFA), Confirmatory Factor Analysis (CFA), and the interpretation of standardized factor loadings. Including relevant references to standard psychometric and statistical modeling literature (e.g., Brown, 2015; Fabrigar & Wegener, 2011; Kline, 2015) would enhance methodological clarity and support the rigor and appropriateness of the analytical approaches used.

Recommended foundational sources the authors could cite:

Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford Publications.

Fabrigar, L. R., & Wegener, D. T. (2011). Exploratory factor analysis. Oxford University Press.

Kline, R. B. (2015). Principles and practice of structural equation modeling (4th ed.). Guilford Publications.

Incorporating these references would significantly enhance the manuscript’s credibility and alignment with standard psychometric practices.

Summary:

The foundational methodological limitations identified in Study 1—particularly the insufficient justification for adopting a one-factor solution despite clear statistical evidence favoring a four-factor structure, inadequate transparency regarding item selection and stability, and reproducibility concerns—my primary recommendation is that the authors revisit and rigorously re-evaluate Study 1 before drawing strong conclusions from subsequent analyses (e.g., Study 2). Addressing these foundational issues will significantly strengthen the psychometric validity, interpretability, and ethical clarity of the findings.

Additionally, while the authors’ commitment to open data practices is commendable, the supplementary code is inadequately documented and lacks necessary details for true reproducibility, significantly limiting independent verification of their results.

For these reasons I recommend major revisions to address the insufficient justification for choosing a one-factor model despite clear statistical evidence supporting a four-factor solution, inadequate documentation and reproducibility of analyses, and overstated interpretations based on preliminary findings. Addressing these issues thoroughly, including incorporating references related to foundational analytics, is critical to ensuring the manuscript’s methodological rigor, interpretability, and ethical clarity.

Enhanced documentation of computational methods is also necessary to ensure data reproducibility.

Consideration:

Although not currently mentioned in the manuscript, the authors might consider leveraging advanced analytical methods, such as machine-learning-driven exploratory techniques or automated cross-validation (e.g., AI-supported factor selection, stability assessments, or iterative modeling), in future validations of their scale. Such AI-enhanced approaches could potentially address methodological shortcomings by improving factor-selection rigor, transparency, reproducibility, and generalizability.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

Development and Validation of the Self-Awareness of Ego-Threatening Biases Questionnaire (SAETBQ)

PONE-D-25-01433R1

Dear Dr. Baumann,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager®  and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Hidenori Komatsu

Academic Editor

PLOS ONE

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I am glad to see that all my comments have been addressed properly. The manuscript should meet the publication standard after proofreading. Well done.

Reviewer #2: The addition of the supplemental documentation and the open source code references (https://osf.io/vhwb5/?view_only=fa592ede448042bda9053cb1b565c5aa) anomalous datapoints provides satisfactory justification for use of the one-factor approach. The models are all statistically sound and logically developed. The justification for selection of the one-factor, once expanded upon, provides an example of how overfitting a model can functionally force statistical confidence, when in reality is diminishes the output. The inclusion of the supplemental an excellent example of how adequate documentation of code structuring, ML model development, ML model choice add the elegant transparency and nuance ML applications can bring.

Code:

bootstrap.R (https://osf.io/z5anc?view_only=fa592ede448042bda9053cb1b565c5aa) - high reproducibility and consistency with table 5,

factor_comparison (https://osf.io/95bc2?view_only=fa592ede448042bda9053cb1b565c5aa) - very high reproducibility with table 4, BN24 exclusion justified

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: No

Reviewer #2: Yes:  AC Demidont

**********