PONE-D-21-05975

VR Realism Scale – revalidation of contemporary VR headsets on a Polish sample

PLOS ONE

Dear Dr. Lipp,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jun 28 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Meng-Cheng Wang

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

If you are reporting a retrospective study of medical records or archived samples, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information.

3. Throughout your manuscript, we noticed several instances of statistical values that are showing up incorrectly and/or are replaced with a blank box, e.g., ln 312. Please correct these in your resubmission.

4. Thank you for providing the following Funding Statement: 

[The work of RS was financed from the science budget (https://www.gov.pl/web/edukacja-i-nauka/diamentowy-grant) for the years 2019–2023 as a research project, ‘The impact of virtually generated characters: human aspects of bots’ (project number: DI2018 015848), under the Diamond Grant program granted by the Ministry of Science and Higher Education of Poland.

Studies A–E were supported by the Polish National Centre for Research and Development (https://www.gov.pl/web/ncbr) under the Widespread Disaster Simulator grant – research and preparation for implementation (project number: POIR.01.01.01-00.0042/16; the Smart Growth Operational Programme, sub-measure 1.1.1. Industrial research and development work implemented by enterprises) received by Nano Games sp. z o.o. AS, PS, NDM, NL, RS received salaries from Nano Games sp. z o.o.

Study F was supported by the Polish National Science Centre (https://www.ncn.gov.pl/) under the grant ‘Wirtualna śmierć realne konsekwencje - efekty śmierci bohatera w grze wideo w świetle Teorii opanowania trwogi’ [ang. Virtual death – real consequences. The effect of virtual agent’s death according to the terror management theory] received by AS (project number: DEC-2018/02/X/HS6/01426 ).

 The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.]. 

We note that one or more of the authors is affiliated with the funding organization, indicating the funder may have had some role in the design, data collection, analysis or preparation of your manuscript for publication; in other words, the funder played an indirect role through the participation of the co-authors.

If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study in the Author Contributions section of the online submission form. Please make any necessary amendments directly within this section of the online submission form.  Please also update your Funding Statement to include the following statement: “The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.”

If the funding organization did have an additional role, please state and explain that role within your Funding Statement.

Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc.  

Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests). If this adherence statement is not accurate and  there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

6. We note that your paper includes detailed descriptions of car licence plate number (Figure 4). As per the PLOS ONE policy (http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research) on papers that include identifying, or potentially identifying, information, the individual(s) or parent(s)/guardian(s) must be informed of the terms of the PLOS open-access (CC-BY) license and provide specific permission for publication of these details under the terms of this license. Please download the Consent Form for Publication in a PLOS Journal (http://journals.plos.org/plosone/s/file?id=8ce6/plos-consent-form-english.pdf). The signed consent form should not be submitted with the manuscript, but should be securely filed in the individual's case notes. Please amend the methods section and ethics statement of the manuscript to explicitly state that the patient/participant has provided consent for publication: “The individual in this manuscript has given written informed consent (as outlined in PLOS consent form) to publish these case details”.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for the opportunity to review “VR Realism Scale — revalidation of contemporary VR headsets on a Polish sample” in the PLOS ONE. This study present validation of the VR Simulation Realism Scale in a sample of Polish participants. I appreciate the authors’ effort of examining psychometric properties of the VR Simulation Realism Scale through five experiments and one online survey, and I have several comments below that the authors may consider.

1. The authors do not make a strong case for the importance of the study (examine psychometric properties of the VR Simulation Realism Scale in a sample of Polish speaking participants). The Introduction does not discuss why this is important.

2. Although the authors discuss immersion, presence and co-presence, the concepts of flow and realism need also properly describe in the Introduction section.

3. In my opinion, I recommend that the authors add the more description of the relationship between VR and criterial measures (e.g., other IVE characteristics) in Introduction section. In particular, the prediction model (i.e., mediation analysis) in the current study.

4. The data analysis strategy is missing in the Material and Methods section, and the related criteria (e.g., internal consistency and CFA) should be described in the Material and Methods (data analysis) not in the Results section.

5. Before conducting the reliability and validity analysis, I suggest that the authors add the item analysis of VR Simulation Realism Scale.

6. Both the model fit of CFA and Bifactor were acceptable, while the factor loadings of several items (e.g., items 1, 6, 10, 14) were unsatisfactory. In general, I’m a little confused why the Sound Realism factor has only one item. Typical, a latent factor requires at least three items to be effectively measured.

7. According to the CFA and Bifactor, the factor loadings of items 1, 6, 10 and 14 were unsatisfactory. The authors need add some description in the Discussion section.

8. It is suggested that the authors further explain and elaborate the relationship between VR and relevant variables (e.g., correlation analysis, mediation analysis) in the Discussion section.

9. The format specification of references.

Other Minor:

1.The expression of IVE and IVEs in the manuscript needs to be consistent.

2. In the table 4, the number of items for Scene Realism factor should be 5.

Reviewer #2: Thanks for the opportunity to review this paper. This study contributes to the virtual experience research by multiple studies with mixed methods. A lot of work has been done to validate the VR Realism Scale and reveal its psychometric properties in Polish context. It is valuable and meaningful. However, I have several concerns, which need to be addressed to improve this paper.

1. It is not easy to follow what your research question is and why it is important in “Introduction”. It seemed that you mentioned it in “Aims and rationale behind our study”, which confused me a little bit regarding the relationship between these two parts. Was the part of “Aims and rationale behind our study” included in the part of “Introduction”? It would be better to reorganize Introduction, with the aim to clarify your research question and why it needs to be addressed logically.

2. As validation of the VR Realism Scale in a different context (i.e., Polish) from where it was developed, is one of the main goals of this paper, measurement invariance across cultures is the key point. What measures did you take to ensure the same meaning of the obtained Polish version scale as that of the German version scale? Who were responsible for back-translating, or comparing the two versions? What were their qualifications for doing so? How did you decide the final version? Could you please provide more details about the scale translation process?

3. A related question, what changes did you make to the original scale? You introduced that “examine the psychometric properties of the scale in the context of state-of-the-art VR technology and modern IVEs, as CAVES and modern headsets differ greatly in terms of the characteristics of the stimuli they deliver (e.g., Mestre, 2017)”. As the technological environments focused are different, what changes did you do for the original one to fit the situation of VR headsets you were specific on?

4. I am not clear about the implication of the bi-factor model result. Is it possible to support the existence of the method effect, as all items were reported by the participates self? And if you could justify that bi-factor model support the one combined structure of simulation Realism, how should we measure this construct, as one general dimension or as multi-dimensions? A related question, why did you treat it as both unidimensional and multi-dimensional in the following analyses?

5. I agree with that it is necessary to examine the relations of simulation realism to other variables. But why did you theorize a mediation model wherein aesthetics as the mediator between simulation realism and positive emotions? Could you please provide more powerful arguments on this study, and explain why this mediation model is needed or helpful to achieve your goal of investigating its relations?

Hope these comments are useful for you.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-21-05975R1VR Realism Scale – revalidation of contemporary VR headsets on a Polish samplePLOS ONE

Dear Dr.Lipp,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Nov 08 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Meng-Cheng Wang

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Both in CFA and Bifactor model, the authors used the fixed load method (e.g., the factor loading for the first item of per factor is fixed to 1), but I’m a little unsure that the standardized factor loading of the first item is also 1, please check it carefully.

Reviewer #2: I am sorry for my delayed comments as it is hard to complete everything in time during the recent surge of pandemic.

I appreciate all your efforts to improve this paper and most of my comments have been addressed satisfactorily. There is only one minor concern: could you please reorganize your introduction session to make it more concise, it is quite long and hard o get the main purpose/ point. For example, you can briefly interpret why you consider “classical aesthetics” as the mediator in introduction session, and then add the “hypotheses session” to provide more arguments for that.

The comments above is for your reference. Good luck!

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

VR Realism Scale – revalidation of contemporary VR headsets on a Polish sample

PONE-D-21-05975R2

Dear Dr. Lipp,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Meng-Cheng Wang

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments: