Same-model and cross-model variability in knee cartilage thickness measurements using 3D MRI systems

Hisako Katano; Haruka Kaneko; Eiji Sasaki; Naofumi Hashiguchi; Kanto Nagai; Muneaki Ishijima; Yasuyuki Ishibashi; Nobuo Adachi; Ryosuke Kuroda; Makoto Tomita; Jun Masumoto; Ichiro Sekiya

doi:10.1371/journal.pone.0324912

Peer Review History

Original SubmissionJanuary 3, 2025
3 Jan 2025 Author Response https://doi.org/10.1371/journal.pone.0324912.r001
25 Feb 2025 Decision Letter - Ewa Tomaszewska, Editor Dear Dr. Sekiya, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Apr 11 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols . We look forward to receiving your revised manuscript. Kind regards, Ewa Tomaszewska, DVM Ph.D Academic Editor PLOS ONE Journal requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, we expect all author-generated code to be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse. 3. In the online submission form, you indicated that [The data used in this study are available from the corresponding author upon reasonable request.]. All PLOS journals now require all data underlying the findings described in their manuscript to be freely available to other researchers, either 1. In a public repository, 2. Within the manuscript itself, or 3. Uploaded as supplementary information. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If your data cannot be made publicly available for ethical or legal reasons (e.g., public availability would compromise patient privacy), please explain your reasons on resubmission and your exemption request will be escalated for approval. 4. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well. 5. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. Additional Editor Comments (if provided): [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? Reviewer #1: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? -->?> Reviewer #1: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available??> The PLOS Data policy Reviewer #1: No ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English??> Reviewer #1: Yes ****** Reviewer #1: The authors of this paper studied same-model and cross-model variability in knee cartilage thickness measurements on 3D images obtained by five manufacturer MRI scanners. It is a niche study that, anyhow, allows to have reference values for intra and inter scanner uncertainties. Sometimes the results are weak: specifically it is obvious that cross-model variability was higher than same-model variability. The text is clear and really well written. The statistical approach is correct. I believe that the title is overfull and should have less information: the number of manufacturers for example is not primary in a title. In the text, including the abstract and the limitations in the Discussion section, should be more stressed that the results are specific for the "3D volume analysis software" that has been used. Minor concerns: 1) in the "Automatic Segmentation" subsection, second line: "using A 3D volume..."; 2) in the "Automatic Segmentation" subsection, fifth line: "from 101 datasets..."; 3) in the "Measurements of Cartilage Thickness" subsection, 18-19 lines should be referenced the Figure 1b. ****** what does this mean? ). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy Reviewer #1: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org https://doi.org/10.1371/journal.pone.0324912.r002
Revision 1
21 Apr 2025 Author Response Reviewer #1: The authors of this paper studied same-model and cross-model variability in knee cartilage thickness measurements on 3D images obtained by five manufacturer MRI scanners. It is a niche study that, anyhow, allows to have reference values for intra and inter scanner uncertainties. Sometimes the results are weak: specifically it is obvious that cross-model variability was higher than same-model variability. The text is clear and really well written. The statistical approach is correct. We sincerely thank the reviewer for their positive assessment of our manuscript, particularly noting that our text is 'clear and really well written' and that our statistical approach is 'correct.' We acknowledge the reviewer's observation that the finding of higher cross-model variability compared to same-model variability may seem intuitive. However, we believe that quantifying these differences provides essential reference values that have not been previously documented in the literature for knee cartilage measurements using modern 3D MRI technology. These reference values serve as critical benchmarks for multi-center studies and longitudinal assessments that utilize different MRI systems. We appreciate the reviewer's recognition of the utility of our work in establishing these reference values for intra and inter-scanner uncertainties. I believe that the title is overfull and should have less information: the number of manufacturers for example is not primary in a title. We have changed from “Five-manufacturer 3D MRI Study on Same-Model and Cross-Model Variability in Knee Cartilage Thickness Measurements” to “Same-Model and Cross-Model Variability in Knee Cartilage Thickness Measurements Using 3D MRI systems”. In the text, including the abstract and the limitations in the Discussion section, should be more stressed that the results are specific for the "3D volume analysis software" that has been used. We have modified the abstract as follows: “Purpose: Magnetic Resonance Imaging (MRI) based three-dimensional analysis of knee cartilage has evolved to become fully automatic. However, when implementing these measurements across multiple clinical centers, scanner variability becomes a critical consideration. Our purposes were to quantify and compare same-model variability (between repeated scans on the same MRI system) and cross-model variability (across different MRI systems) in knee cartilage thickness measurements using MRI scanners from five manufacturers, as analyzed with a specific 3D volume analysis software. Methods: Ten healthy volunteers (eight males and two females, aged 22-60 years) underwent two scans of their right knee on 3T MRI systems from five manufacturers (Canon, Fujifilm, GE, Philips, and Siemens). The imaging protocol included fat-suppressed spoiled gradient echo and proton density weighted sequences. Cartilage regions were automatically segmented into 7 subregions using a specific deep learning-based 3D volume analysis software. This resulted in 350 measurements for same-model variability and 2,800 measurements for cross-model variability. Results: For same-model variability, 82% of measurements showed variability ≤0.10 mm, and 98% showed variability ≤0.20 mm. For cross-model variability, 51% showed variability ≤0.10 mm, and 84% showed variability ≤0.20 mm. The mean same-model variability (0.06 ± 0.05 mm) was significantly lower than cross-model variability (0.11 ± 0.09 mm) (p<0.001). Conclusion: This study demonstrates that knee cartilage thickness measurements exhibit significantly higher variability across different MRI systems compared to repeated measurements on the same system, when analyzed using this specific software. This finding has important implications for multi-center studies and longitudinal assessments using different MRI systems and highlights the software-dependent nature of such variability assessments.” We have also modified the limitations as follows: “A few limitations should be considered when interpreting the results of this study. First, it is important to emphasize that our findings are specific to the particular 3D volume analysis software (SYNAPSE 3D) that was used in this study. Different segmentation algorithms or analysis software might yield different variability patterns across manufacturers. The software-specific nature of our results limits their direct applicability to studies using alternative analysis platforms. Second, the sample size was limited to only 10 subjects, which may restrict the generalizability of our findings. Third, our study focused solely on healthy volunteers without knee complaints. A more diverse cohort with a balanced male-to-female ratio and including subjects of various ages and with mild to severe OA could provide more comprehensive and clinically relevant results. The variability in cartilage thickness measurements might differ in patients with osteoarthritis or other knee conditions due to altered cartilage states. Finally, our assessment was limited to two scans per subject, providing only a short-term evaluation. A longitudinal study with multiple time points could offer insights into long-term variability and the stability of measurements over time, potentially revealing how variability across MRI systems might change with disease progression.” Minor concerns: 1) in the "Automatic Segmentation" subsection, second line: "using A 3D volume..."; We have corrected as follows: “MRI analyses using deep learning–based segmentation were performed using a 3D volume analysis software (SYNAPSE 3D [Japanese product name: SYNAPSE VINCENT], Collaborative version 6.7, Fujifilm Corporation, Tokyo, Japan).” 2) in the "Automatic Segmentation" subsection, fifth line: "from 101 datasets..."; We have corrected as follows: “The automatic segmentation algorithm was trained on SPGR and PDW images from on 101 datasets obtained from Philips, nine from Canon, five from GE, two from Fujifilm, and none from Siemens.” 3) in the "Measurements of Cartilage Thickness" subsection, 18-19 lines should be referenced the Figure 1b. We have corrected as follows: “The patellar cartilage was projected so that the area reached its maximum, and the ROI was automatically drawn as a closed curve line (Figure 1b) (10).” Attachments Attachment Submitted filename: 3D-MRI Ethics Approval M2021-291 Plos 250102 Japanese.pdf https://doi.org/10.1371/journal.pone.0324912.r003
4 May 2025 Decision Letter - Ewa Tomaszewska, Editor Same-model and cross-model variability in knee cartilage thickness measurements using 3D MRI systems PONE-D-25-00129R1 Dear Dr. Ichiro Sekiya, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Ewa Tomaszewska, DVM Ph.D Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author Reviewer #1: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions??> Reviewer #1: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? -->?> Reviewer #1: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available??> The PLOS Data policy Reviewer #1: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English??> Reviewer #1: Yes ****** Reviewer #1: The authors of this paper have answered all my comments and fully addressed my concerns updating the text accordingly. ****** what does this mean? ). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy Reviewer #1: No ******** https://doi.org/10.1371/journal.pone.0324912.r004
Formally Accepted
Acceptance Letter - Ewa Tomaszewska, Editor PONE-D-25-00129R1 PLOS ONE Dear Dr. Sekiya, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset You will receive further instructions from the production team, including instructions on how to review your proof when it is ready. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few days to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Professor Ewa Tomaszewska Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0324912.r005

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .