PONE-D-23-15143A webcam-based machine learning approach for three-dimensional range of motion evaluationPLOS ONE

Dear Dr. Wang,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

Thank you for submitting your manuscript to PLOS ONE. The reviewers have made a number of helpful suggestions to consider as you revise your manuscript. Please be sure to address all reviewer comments in a point-by-point response document. We look forward to receiving your revised manuscript!

==============================

Please submit your revised manuscript by Aug 18 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Ryan T. Roemmich

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, all author-generated code must be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse.

3. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match. 

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

4. Thank you for stating the following in the Acknowledgments Section of your manuscript: 

"This study was supported by the National Institute on Minority Health and Health Disparities (STTR: #1R41MD015689-01), and the US Department of the Treasure and Wyoming Health & Bioscience Innovation Hub (CARES-HUB-MOVE#1040)."

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. 

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: 

"The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

5. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. 

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

6. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well. 

Additional Editor Comments:

Thank you for submitting your manuscript to PLOS ONE. The reviewers have made a number of helpful suggestions to consider as you revise your manuscript. Please be sure to address all reviewer comments in a point-by-point response document. We look forward to receiving your revised manuscript!

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Summary:

In this work, the authors evaluate a webcam-based method for measuring range of motion (ROM) including flexion, extension, and rotation of the neck, back, and upper and lower extremities. The authors compare their proposed method to a motion-capture system (OptiTrack), evaluating the ICC, standard error of measurement (SEM), and minimal detectable change (MDC) within each system, as well as between the systems. The methods are evaluated on a cohort of 25 university students.

General Comments:

The paper is well-written and presents interesting results with respect to the performance of a webcam-based method for ROM assessment. The authors note good to excellent ICC within multiple trials of the webcam method for a given participant. However, the ICC between the webcam and OptiTrack varied significantly by each joint. The authors attribute this to differences in keypoint/marker placement between the methods, and concerns with lack of contrast due to the dark clothing that must be worn during motion capture.

Specific Comments:

1. Please provide more detail in the Abstract with respect to the method, the joints assessed, and specific key findings.

2. Please do not capitalize “Pandemic” throughout the manuscript.

3. Lines 64 – 66: Consider removing CNNs from this sentence and discussing this separately. Currently the other approaches listed in this sentence are different modalities for collecting data, whereas CNNs can be used in conjunction with any of these as a means to process and form meaningful conclusions from the collected data.

4. Lines 69 – 70: Please also include the need for a large physical space to perform motion capture as a limitation of this approach.

5. Lines 70 – 71: Please change “In recent years, the machine learning-based human pose estimation has seen significant development.(19–21)” to “In recent years, the machine learning-based human pose estimation field has seen significant development.(19–21)” or similar.

6. Lines 83 – 89: Please also add gait assessment as a field where the use of human-pose estimation has been explored.

7. The study in this work focused on multiple joints in the upper body: J. Fan et al., “Reliability of a human pose tracking algorithm for measuring upper limb joints: comparison with photography-based goniometry,” BMC Musculoskelet Disord, vol. 23, no. 1, p. 877, 2022, doi: 10.1186/s12891-022-05826-4.

8. Section 2.2: Please provide the name of the motion capture system in this section, as well as the sampling frequency of both the motion capture system and the webcam.

9. Methods - General: Was the webcam placed a prescribed distance from the participant for each movement?

10. Table 1: Please specify the orientation of the camera/plane from which the video was taken for each movement.

11. Lines 172 – 174: Please justify why a different cut-off frequency was used for the webcam and motion capture system.

12. Lines 236 – 238: Were the inter-rater ICC results significantly different when the measurements from BlazePose and OptiTrack were not averaged over the 3 trials? If possible, it may be valuable to investigate whether the process of averaging has affected the results in a significant way.

13. Lines 238 – 240: Please review this sentence, as it does not appear to be complete.

14. Table 2: Please provide units in the table headings where applicable (ie for SEM and MDC).

15. Lines 270 – 274: Was this hypothesis supported by a higher percentage of timesteps where visibility <0.5 for the movements with lower ICC? It may be valuable to provide an summary of the percentage of timesteps where the visibility was less than the 0.5 cut-off (in a supplemental document or the main manuscript).

16. Section 3.3: It would also be valuable to see correlations by-joint (potentially in the supplemental material)

17. Lines 342 – 343: While the mean slope is 1.01, the range is [0.83 – 1.13] which suggests the mapping between the two modalities is not always “one-to-one” as the authors suggest. Presenting an analysis by-joint may provide further insight with respect to whether there are any joints where the mapping between the two systems is better or worse.

18. Figure 4a): Please ensure that the right-most edge of the figure is not cropped out/obstructed.

19. Lines 383 – 384: Reference (41) focuses solely on the cervical joint. It is unclear whether the MDC of 10 degrees applies to all joints studied in this work.

20. Lines 384 – 387: “Careful examination” of the movements with lower inter-rater reliability as claimed in this sentence has not been presented in this manuscript. Further examination to support this hypothesis is required to make this claim. Please reword this sentence if further analysis on this hypothesis is not possible.

21. Line 416: “appropriate camera placement” is a key component of the webcam-based method presented in this work. Please provide additional detail as to the height and distance of the camera to the participant, as well as how the videos were framed for each movement.

22. Line 425: Based on the results in Table 2, it is too strong of a claim to state that there was high inter-rater reliability across all joints. Please rephrase this to acknowledge that inter-rater reliability was low for some joints.

Reviewer #2: Overall Impression:

- This study explored the use of a single-camera approach to obtaining human biomechanics by using a machine-learning solution (Blazepose) based on human pose detection. The reliability of this approach is studied in a sample of 25 adult participants, and the accuracy of the approach is compared to that of a marker-based motion capture system. While I’m excited to see validation studies on such systems, I recommend further revisions to align author’s conclusions with the questions tested in this study.

Major issues:

- Recommend caution with claim of cost-effectiveness (and accessibility) as this does not appear to be a factor tested in this paper. Without explicit testing, the cost/accessibility of a device-based Blazepose approach to ROM measurement would seem more than that of a simple goniometer.

- There is no limitations section in paper.

- Statistical methods seem to be duplicative. For example, the authors use inter-rater reliability, linear regression and root-mean-square-error to describe the accuracy of Blazepose method versus motion capture method.

Minor issues:

- Beginning line 34, the abstract would benefit from a concise statement of actual findings regarding reliability and accuracy rather than qualitative statements alone.

- Line 70, authors state that technical proficiency is a barrier to motion capture. Would there not be some technical barriers to implementing the Blazepose solution? While hopefully less barriers, consider acknowledging (in a limitations section) that some technical hurdles still remain for this method.

- Typo on line 84, “have been used” instead of “has been used”

- Typo on line 87, “examines” instead of examine

- Line 94, while motion capture is an acceptable comparator, the authors have stated the goniometer is one of the most common ways to measure ROM. I would then expect to see this as a comparator in the study.

- Line 95, the last sentence of section claims Blazepose as an alternative to traditional ROM evaluation. Are authors referring to the goniometer as traditional method? How have the authors tested the accessibility of Blazepose versus a goniometer? This sentence would benefit from revision, and contents of sentence would be better suited in results section.

- Line 103, the methods section would benefit from more details on participant demographics. Were all participants healthy / neurotypical? Were any neurodivergent populations included? Were any amputees included?

- Typo on line 127, “were used” instead of “was used”

- Line 130, if webcam video of participants, was face occlusion used to prevent identification? Based on Figure 1, the red dots provide some occlusion but I would be concerned that the remainder of visible face could potentially be recognized and person identified.

- Line 143, please clarify whether the ROM evaluation movements are based on a standard or on some other citable reference. Also, it might benefit the audience to explain/justify why more distal joint movements were excluded, e.g. wrist and ankle ROM.

- Line 152, the standing and repositioning required for the webcam could be quite challenging for some participants, especially individuals who are physically frail or neurodivergent. These populations often require therapy interventions to address function/quality of life. While I’m excited to see these pose detection systems and expect future iterations will be more robust to human movement in-the-wild, the current system described does have limitations which are important for clinicians to understand.

- Line 157, for repeated measures of each movement, were these measures all performed during the same session? If so, was there a prescribed time period between measures?

- Line 173, please provide some explanation or reference regarding the choice of pre-processing filter designs. Would cutoff frequency of 5Hz risk filtering out the frequency of some human movements?

- Beginning at Line 198, the methods to calculate ROM seem overly complex and the justification is confusing. Using motion capture or the Blazepose method, I would expect these to yield joint locations in 3D, which could then be used to define vectors (e.g. for elbow angle, a vector from shoulder-to-elbow would exist and a vector from elbow-to-wrist would exist). The authors could then determine ROM by calculating the angle between vectors at every instance in time (as the authors later describe). While the pivot joint is rotating and translating with time as the authors depict in Figure 2, the ability to determine the angle at each instance in time would make this rotation/translation seemingly a non-factor. Please clarify further (or highlight references) to support using best-fit planes and joints centroids to calculate the ROM.

- Line 327, the authors report that compression around the acromion caused the Optitrak shoulder location to be perturbed. Given that the 3D coordinate data is available, please quantify the perturbation for the reader. Based on Figure 3a, this perturbation visually appears quite small, and its contribution to the higher interrater reliability in the shoulder flexion ROM would also seem small. Likewise, as 3D coordinate data is available, please quantify the “lack of perturbation” of the shoulder marker during shoulder extension movement.

- Line 412, the authors state the webcam approach is affordable and accessible, but these must be interpreted relatively. Do the authors suggest webcam is more affordable/accessible than goniometer? Or motion capture? Furthermore, as affordability/accessibility were not tested in the described study, I suggest these statements be revised.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Justin Huber

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-23-15143R1A webcam-based machine learning approach for three-dimensional range of motion evaluationPLOS ONE

Dear Dr. Wang,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================Please provide the minor revisions as suggested by Reviewer 1.==============================

Please submit your revised manuscript by Nov 18 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Ryan T. Roemmich

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

Please make the following revision suggested by Reviewer 1: The title and text of figure 5 say "shoulder flexion", but the image and in-text discussion is about elbow flexion. Please fix.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: the title and text of figure 5 say "shoulder flexion", but the image and in-text discussion is about elbow flexion. Please fix this before publication.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

A webcam-based machine learning approach for three-dimensional range of motion evaluation

PONE-D-23-15143R2

Dear Dr. Wang,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Ryan T. Roemmich

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments: