https://orcid.org/0000-0003-1313-8044
Peer Review History
| Original SubmissionDecember 18, 2024 |
|---|
|
PONE-D-24-54281Protocol for Evaluation of a Virtual Wheelchair Simulator in Assessing Mobility Skills and Cognitive Abilities in Diverse Populations: A Multicentric Mixed-Methods Pilot StudyPLOS ONE Dear Dr. Pereira Salgado, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Congratulations on this initiative! This topic is of great importance to rehabilitation teams. Please see the reviewers' comments and questions below Please submit your revised manuscript by Mar 24 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:
If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols . We look forward to receiving your revised manuscript. Kind regards, Rodrigo Rodrigues Gomes Costa, PhD Academic Editor PLOS ONE Journal Requirements: 1. When submitting your revision, we need you to address these additional requirements. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 2. When completing the data availability statement of the submission form, you indicated that you will make your data available on acceptance. We strongly recommend all authors decide on a data sharing plan before acceptance, as the process can be lengthy and hold up publication timelines. Please note that, though access restrictions are acceptable now, your entire data will need to be made freely accessible if your manuscript is accepted for publication. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If you are unable to adhere to our open data policy, please kindly revise your statement to explain your reasoning and we will seek the editor's input on an exemption. Please be assured that, once you have provided your new statement, the assessment of your exemption will not hold up the peer review process. 3. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please move it to the Methods section and delete it from any other section. Please ensure that your ethics statement is included in your manuscript, as the ethics statement entered into the online submission form will not be published alongside your manuscript. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions? The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field. Reviewer #1: Yes Reviewer #2: Partly ********** 2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses? The manuscript should describe the methods in sufficient detail to prevent undisclosed flexibility in the experimental procedure or analysis pipeline, including sufficient outcome-neutral conditions (e.g. necessary controls, absence of floor or ceiling effects) to test the proposed hypotheses and a statistical power analysis where applicable. As there may be aspects of the methodology and analysis which can only be refined once the work is undertaken, authors should outline potential assumptions and explicitly describe what aspects of the proposed analyses, if any, are exploratory. Reviewer #1: Partly Reviewer #2: Partly ********** 3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable? Descriptions of methods and materials in the protocol should be reported in sufficient detail for another researcher to reproduce all experiments and analyses. The protocol should describe the appropriate controls, sample size calculations, and replication needed to ensure that the data are robust and reproducible. Reviewer #1: Yes Reviewer #2: No ********** 4. Have the authors described where all data underlying the findings will be made available when the study is complete? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception, at the time of publication. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above and, if applicable, provide comments about issues authors must address before this protocol can be accepted for publication. You may also include additional comments for the author, including concerns about research or publication ethics. You may also provide optional suggestions and comments to authors that they might find helpful in planning their study. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Congratulations on choosing this topic. It is an important issue that could expand the use of powered mobility wheelchairs among patients with cognitive impairments. The study protocol is well-structured. However, some refinements could enhance its robustness, validity, and applicability. Below are some methodological recommendations. 1. Study design: - (Line 76 x 115) Is it a pilot study or a validation study? 2. Individual variability: - (Line 151) What assurance do you have in a convenience sample that you will have an adequate group to test the validity of the simulator (both for the use of the powered wheelchair and for cognitive impairments)? - (Line 154) The variability of age in a study can be considered either selection bias or a confounding bias. If the sample of participants does not adequately represent the age distribution of the target population, this could affect the validity of the results, leading to selection bias. On the other hand, if age is not properly controlled as a confounding variable, it may influence the results in a way that distorts the relationship between the analyzed variables, constituting confounding bias - (Line 154) Address potential bias and individual variability given that participants (especially wheelchair users) may have varying levels of experience and cognitive/physical conditions. Individual variability could influence the results. Implement stratification methods to categorize participants based on their experience level in wheelchair operation and cognitive function. This would minimize bias and ensure a more homogeneous comparison across groups. - (Line 128 x 139 x 153) Wouldn't it be more appropriate to have an experimental group that uses a wheelchair and has cognitive impairments, and a matched control group that uses a wheelchair without cognitive impairments, for better outcome validation? - (Line 156) Did the users operate the powered wheelchair only with their hands (manual joystick)? Or will the sample also include users who operate it with their head, chin, or foot? It is important to describe and select the sample that is expected. - (Line 164) Could an individual with unrecognized visual and/or auditory impairments interfere with the data collection? Wouldn't these impairments be an exclusion criterion? Wouldn't it be necessary to evaluate these individuals for these criteria. 3. Sample size: - (Line 175) Provide a more detailed justification for sample size. The study proposes a sample size of 10 to 15 participants per group, citing recommendations from pilot studies. Consider including a preliminary statistical analysis or a power calculation based on the expected sensitivity and specificity of the simulator compared to the reference tests (WST, PMRT, MoCA). This would ensure that the chosen sample size is adequate for testing the study hypotheses with sufficient statistical power. 4. Simulator - (Line 92) It would be interesting to add a photo of the simulator with its components. - (Line 271) Define clear success criteria for the simulator. The protocol states that the simulator’s performance will be compared to standard clinical tests, but specific cut-off values for validation are not clearly defined. Establish objective performance thresholds for simulator validation. 5. Structure and grammar - Line 95: space before “,”. - Line 102: space after “PMRT”. - Line186: space after “IPQ”. - Line 386: “The”. The study is well-designed, but incorporating statistical rigor, bias control, and well-defined validation criteria would further strengthen its findings. These refinements will help ensure that the study produces clinically relevant and generalizable results. Reviewer #2: The proposed protocol aims to develop and validate a protocol for assessing the efficiency of metrics within a virtual wheelchair simulator, which is also intended to evaluate mobility and cognitive skills in wheelchair users. This protocol holds significant clinical relevance given the increasing use of technology in rehabilitation. Having a validated tool for wheelchair training that also assesses mobility and cognitive abilities will enhance the safety and accuracy of powered wheelchair prescriptions for individuals with disabilities. However, several key aspects of the manuscript require further clarification. Below are some considerations for the authors: • The manuscript is overly lengthy, with certain ideas repeated throughout. I recommend a thorough revision to make the text more concise and improve readability. • The study aims to develop and validate a protocol for evaluating the efficiency of the simulator’s metrics. However, what is the study’s primary outcome? Is it validating the simulator’s ability to assess mobility and cognitive skills, or is it a pilot study designed to develop an evaluation protocol for future validation research? If it is a pilot study, it would be beneficial to explicitly describe how the authors plan to refine the protocol for a subsequent validation study. • The authors propose using multiple tools to assess the simulator’s efficiency, including the Wheelchair Skill Test (WST), Power Mobility Road Test (PMRT), Montreal Cognitive Assessment (MoCA), physiological data, cognitive workload, usability, immersion, and emotional response. These variables will be analyzed across three groups (wheelchair users, non-disabled individuals, and healthcare professionals) using distinct statistical comparisons. However, the manuscript does not clearly specify the type of validation being employed (e.g., construct, concurrent, predictive), the reference values that would determine the simulator’s validity, and which variables will be used for its validation. Introduction: • Line 48: The phrase “in this study” appears, but it is unclear which study the protocol refers to. It initially seems to describe the simulator’s assessment protocol. However, in my suggestion, the introduction should primarily establish the scientific basis for the tools used and the rationale behind the study. A description of the protocol should be placed in the methods section or in the final paragraph of the introduction, where the study objectives are presented. • Line 62: As mentioned earlier, the explicit justifications for the pilot protocol could be better positioned in the methods section or the final paragraph of the introduction. Materials and Methods: • Lines 87–88: As previously noted, the manuscript suggests that the goal is to validate the virtual simulator as a reliable tool for wheelchair training and assessment. However, earlier in the text, it is also stated that the objective is to “develop a protocol for validation,” which implies that a protocol will be designed for future validation. The study’s objective must be clearly defined. Furthermore, reliability is another psychometric variable that is not explored in the study protocol. • Lines 105–106: The manuscript indicates that the pilot study aims to validate the simulator’s ability to estimate PMRT, WST, and MoCA scores. However, validation cannot be achieved in a pilot study alone; it requires an adequate sample size for statistical validation. Additionally, the text does not specify how the simulator will generate scale scores. Will predictions be based on algorithms within the simulator, or will users perform tasks similar to those required for scale scoring? • Lines 107–110: Why were non-disabled individuals with no wheelchair experience chosen as the baseline for assessing system sensitivity? Even without disabilities, individuals unfamiliar with powered wheelchairs may still experience difficulties using them. • Lines 128–163: The age range (18–80 years) is repeated multiple times throughout these paragraphs. In my opinion, this information does not need to be reiterated. • Lines 129–130: What criteria will the authors use to differentiate between novice and experienced wheelchair users? • Lines 140–142: The term “healthy participants” is outdated. I recommend replacing it with “non-disabled individuals.” • Lines 165–167: What are the severe impairments and medical contraindications that would exclude participants? Providing examples would be beneficial, as the simulator is intended for individuals with physical and cognitive disabilities. • Lines 164–171: Some exclusion criteria merely restate the inclusion criteria in reverse. If participants who do not meet the inclusion criteria are already excluded, redundant criteria should be removed. Instead, the exclusion criteria should highlight relevant conditions specific to the study population. • Lines 174–180: The sample size justification is based on the authors’ assertions rather than a specific statistical power calculation. Since this is a validation study, an appropriate sample size calculation should be conducted to ensure adequate statistical power. • Lines 181–184: The study will conduct assessments before, during, and after simulator use. It is essential to reference Table 1, which outlines each step of the study. • Lines 192–194: The manuscript mentions a grounded theory approach and triangulation methods but does not explain how these methodologies will be implemented. Additionally, no references are cited to support these methods. • Lines 214–216 & 265–270: Who will administer the WST and MoCA assessments? Will they be conducted by healthcare professionals before the test, or will they be self-reported? • Lines 272–278: What will the cutoff points differentiate? Will they classify participants into categories such as “good,” “moderate,” and “poor” mobility, or will they distinguish between individuals with and without wheelchair skills? This needs to be explicitly stated. • Lines 284–290: Several claims in this paragraph lack citations. • Lines 300–305: How will artificial intelligence be used in the project? It is unclear which scores will be estimated and which will be predicted. • Supplementary Materials: The manuscript does not mention the use of STROBE as a quality assessment tool, yet it is included as an appendix. Will the study use two quality checklists (STROBE and STARD)? ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy . Reviewer #1: Yes: Ana Claudia Garcia Lopes Reviewer #2: No ********** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step. |
| Revision 1 |
|
Protocol for Evaluation of a Virtual Wheelchair Simulator in Assessing Mobility Skills and Cognitive Abilities in Diverse Populations: A Multicentric Mixed-Methods Pilot Study PONE-D-24-54281R1 Dear Dr. Salgado, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Rodrigo Rodrigues Gomes Costa, PhD Academic Editor PLOS ONE Comments to the Author 1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions? The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field. Reviewer #2: Yes ********** 2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses? The manuscript should describe the methods in sufficient detail to prevent undisclosed flexibility in the experimental procedure or analysis pipeline, including sufficient outcome-neutral conditions (e.g. necessary controls, absence of floor or ceiling effects) to test the proposed hypotheses and a statistical power analysis where applicable. As there may be aspects of the methodology and analysis which can only be refined once the work is undertaken, authors should outline potential assumptions and explicitly describe what aspects of the proposed analyses, if any, are exploratory. Reviewer #2: Yes ********** 3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable? Descriptions of methods and materials in the protocol should be reported in sufficient detail for another researcher to reproduce all experiments and analyses. The protocol should describe the appropriate controls, sample size calculations, and replication needed to ensure that the data are robust and reproducible. Reviewer #2: Yes ********** 4. Have the authors described where all data underlying the findings will be made available when the study is complete? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception, at the time of publication. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #2: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #2: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above and, if applicable, provide comments about issues authors must address before this protocol can be accepted for publication. You may also include additional comments for the author, including concerns about research or publication ethics. You may also provide optional suggestions and comments to authors that they might find helpful in planning their study. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #2: I commend the authors for the revisions made to the manuscript. I believe that the main objective of the study, as well as the methodological procedures, are now more clearly defined. Below, I offer some specific suggestions related to grammar and writing: • Line 66: It is recommended to include the authors' names instead of using citation [18] as a noun. • Lines 215–219: As it is now clear that the study is intended to be a pilot study rather than a validation study, I suggest removing this paragraph regarding sample size calculation, as it will be more relevant in a future full validation of the tool. • Line 203: Please review the expression “ep to,” which appears to be a typographical error. • Line 363: The phrase “such as” is duplicated and should be corrected to improve clarity and readability. ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy . ********** |
| Formally Accepted |
|
PONE-D-24-54281R1 PLOS ONE Dear Dr. Pereira Salgado, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset You will receive further instructions from the production team, including instructions on how to review your proof when it is ready. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few days to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Professor Rodrigo Rodrigues Gomes Costa Academic Editor PLOS ONE |
Open letter on the publication of peer review reports
PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.
We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.
Learn more at ASAPbio .