PONE-D-21-13568

Accuracy of Kinovea software in estimating body segment movements during falls captured on standard video: effects of fall direction, camera perspective and video calibration technique

PLOS ONE

Dear Dr. Shishov,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 22 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

J. Lucas McKay, Ph.D., M.S.C.R.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

3.  We note that Figure 1 includes an image of a participant in the study. 

As per the PLOS ONE policy (http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research) on papers that include identifying, or potentially identifying, information, the individual(s) or parent(s)/guardian(s) must be informed of the terms of the PLOS open-access (CC-BY) license and provide specific permission for publication of these details under the terms of this license. Please download the Consent Form for Publication in a PLOS Journal (http://journals.plos.org/plosone/s/file?id=8ce6/plos-consent-form-english.pdf). The signed consent form should not be submitted with the manuscript, but should be securely filed in the individual's case notes. Please amend the methods section and ethics statement of the manuscript to explicitly state that the patient/participant has provided consent for publication: “The individual in this manuscript has given written informed consent (as outlined in PLOS consent form) to publish these case details”.

If you are unable to obtain consent from the subject of the photograph, you will need to remove the figure and any other textual identifying information or case descriptions for this individual.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This study sought to determine the accuracy with which an open source biomechanical software package, Kinovea, could be used to estimate the kinematics of laboratory induced falls with respect to a gold standard 3D motion capture system. This is an interesting paper, which pending revision, may lower the bar for entry into the analysis of real-world falls. This is a big positive for those of us involved in falls research. The authors therefore are to be commended for taking on this project, and doing so with such breadth of independent variables. That being said, my enthusiasm for the manuscript was somewhat tempered by a variety of concerns regarding methodology, as well as presentation and interpretation of study results. These included but were not limited to the absence of any recommendations for the use of Kinovea when analyzing falls, complex figures (which may be better suited as tables given the volume of data), and the restatement of results within the discussion.

Major

1. It would be extremely helpful to the reader, and ease interpretation as well as future application, if the authors provided guidelines or recommendations on the use of Kinovea for the assessment of falls videos based on the observed RMSE values (i.e., when is the RMSE ok versus "too much"?). For example, should Kinovea be used for all fall directions? Both position and velocity data? All body segments? While this is hinted at in the conclusion, it is not well described with specifics in the discussion.

2. The authors describe difficulty with differentiation of position signals to velocity data. Did the authors consider or might the consider using a Svelsky-Golay filter? We have found this quite effective in calculating velocity signals from position data with minimal noise.

3. Figures 4 through 8 are a lot to take in. Given the density of the data would this data be better presented as tables?

4. Much of the discussion, and especially the first 2-3 sentences of nearly every paragraph, consists of a restatement of the results. This seems unnecessary. The discussion could be better focused on how and when to use Kinovea for the analysis of falls based on comment 1 above. This may align the discussion with the results quite well.

5. Could the authors provide a justification for the use of Kinovea versus some other open source software package like DeepLabCut that has become popular for the analysis of video data. Is a comparison of Kinovea to DeepLabCut necessary?

Minor

1. Could the authors clarify why they chose to calculate segment rather than anatomical joint angles?

2. Could the authors clarify whether reflective markers used in the 3D analysis were or were not used when processing the Kinovea data? If so, what implications might this have for the application of Kinovea to video without passive reflective markers?

3. In the methods the authors describe an assessment of inter-rater reliability using RMSE. It is not immediately clear if this is an appropriate method by which to assess inter-rater reliability. Should the authors be determining inter-rater reliability using intra-class correlation coefficients (ICC)? The following citation may provide guidance in this decision.

Weir JP. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond Res. 2005;19(1):231-240.

Reviewer #2: Accuracy of Kinovea software in estimating body segment movements during falls captured on standard video: effects of fall direction, camera perspective and video calibration technique

Summary of reviewer comments: The manuscript focuses on a significant and clinically-relevant problem –measurement of movement quality during falls in real-world settings. The study methods comprised laboratory-based falling experiments, and comparison of full body kinematics (linear and angular positions and velocities) measured from 3D motion capture to those estimated by Kinovea 2D digitization software from standard surveillance video cameras. The effects of several methodological factors on the accuracy of Kinovea were evaluated, including fall direction, camera angle, filtering cut-off frequency, and calibration techniques. The results are comprehensive and presented clearly. However, the rationale for the study and comparison of current work in light of previous literature needs strengthening. The study is limited by a very small sample size (N=3), which may be caused by the unique methodology employed but needs justification. The discussion can benefit from reorganization and editing.

Specific comments are listed below:

Abstract – The statements regarding the results can benefit from inclusion of some numbers, e.g. “When compared to 90 deg, a 60 deg camera angle yielded less accurate horizontal velocities (include magnitude or % decrease in error here). The conclusion of the abstract could be edited to be more specific and to better discuss the implications of the results.

Introduction –

“Improved understanding of the kinematics of real-life falls should help to inform efforts for injury prevention” – it would strengthen the introduction if the authors could provide details or examples of how measuring kinematics during falls can inform fall prevention programs.

“compared to 3D motion capture, Kinovea estimated hip, knee, and ankle angles during walking with errors less than 5 deg” – please clarify if this stated accuracy or error is for 2-D or sagittal plane angles? Also, please refrain from using “deg” instead of degrees throughout the text of the manuscript.

Because the manuscript has several objectives related to evaluation of Kinovea for measurement of kinematics during falls, it would improve clarity if the authors could edit the last para of the Introduction to clearly list the study objectives (in order of importance), and also provide the rationale or importance of each stated objective. Additionally, if possible, a couple sentences about previous studies that may have addressed the same or similar questions would be beneficial for the reader.

Methods –

The study involves a very small sample size (3 individuals). Please clarify the rationale for this sample size in the methods section, and a statement about the sample size as a limitation in the Discussion section.

The methods used for the 2 different camera calibration methods should be described in more detail, and appropriate supportive references provided, if available.

There may be a formatting error in Table 1 (columns on the far right appear truncated/cropped).

“We used one-way ANOVAs to examine how the accuracy of Kinovea signals associated with Kinovea filter cut-off frequency, camera angle, fall direction, body part, and calibration technique”- this description is somewhat confusing as written. Was the ANOVA used to test associations or the effect of these methodological variables on errors? Were separate ANOVAs performed for each variable (e.g. camera angle, filter settings, etc)? Please edit to improve clarity. Similarly, the rationale for repeated measures comparisons for the same fall could be better clarified.

Results and Discussion –

The Discussion can be strengthened by including more detailed interpretation of the study findings, placing the current results in context of previous literature, as well as methodological and clinical implications or insights gained from the study.

The discussion should also include potential reasons for instances where fairly high errors were observed (e.g. Figure 7 C shows high angular position errors for certain segments).

Figure 1 could benefit from additional labels to show the relevant objects / landmarks in each panel.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Accuracy of Kinovea software in estimating body segment movements during falls captured on standard video: effects of fall direction, camera perspective and video calibration technique

PONE-D-21-13568R1

Dear Dr. Shishov,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Manabu Sakakibara, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No