A novel application of neural networks to identify potentially effective combinations of biologic factors for enhancement of bone fusion/repair

Albert T. Anastasio; Bailey S. Zinger; Thomas J. Anastasio

doi:10.1371/journal.pone.0276562

Peer Review History

Original SubmissionJune 9, 2022
8 Aug 2022 Decision Letter - Kathiravan Srinivasan, Editor PONE-D-22-16681A Novel Application of Neural Networks to Identify Potentially Effective Combinations of Biologic Factors for Enhancement of Bone Fusion/RepairPLOS ONE Dear Dr. Anastasio, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. ACADEMIC EDITOR: Please revise the manuscript based on the reviewers suggestions and comments. Please submit your revised manuscript by Sep 22 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Kathiravan Srinivasan Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf. 2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, all author-generated code must be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse. 3. Please amend either the abstract on the online submission form (via Edit Submission) or the abstract in the manuscript so that they are identical. 4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information. 5. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Thank you very much for the chance to review this manuscript. The authors extracted literature data on bone healing to train DL models for prediction modeling with regard to treatment success. I read the manuscript with great interest and think it provides an innovative way of processing the current data patterns in the literature to improve patient treatment outcomes. Although the available data points on this topic are small at the moment and the provided models’ capacities are insufficient for accurate predictions, I think this technique could be used in the future when more data is available in literature or other fields where sufficient data is already available. I recommend publishing this work as I believe it significantly contributes to the literature. I only have some minor comments which I recommend addressing before publication: 1)how was the literature search performed? The process of data collection (literature search) should be provided in more detail (e.g., search algorithm, which databases), as the results are based on them. 2)P6 “The analysis indicates that combinations of BMP2, BMP7, and OG, perhaps including ES or BMAC, each at the high end of their ranges as reported in relevant studies, should outperform combinations that lack those components. “ Which relevant studies? They should be cited after this statement. 3)please provide the hardware and software used for model building and training for reproducibility of methods provided 4)usually, there is also a validation dataset used that the algorithms have not seen before, resulting in a train, test, and validation dataset. As the testing data will also be used in the hyperparameter tuning process (to evaluate the best parameters), it is recommended by available guidelines on the reporting of machine and deep learning algorithms to include this type of internal validation. However, I see that with the limited data points, this seems to be not feasible. I recommend discussing this point in 1-2 sentences in the discussion section. Reviewer #2: i. In abstract add more regarding the novel contribution of the manuscript. ii. Literature review is shallow in context of various modeling technics. iii. For Data Driven AI model specially Deep Learning Model needs large data, tyr to research and add more. iv. Novelty of the work in terms of comparison to existing comparable models is missing, you can refere article for refrence,Automated skull damage detection from assembled skull model using computer vision and machine learning.Int. j. inf. tecnol. 13, 1785–1790 (2021). https://doi.org/10.1007/s41870-021-00752-5 v.The running time of the proposed model is not counted in the experiment. Time complexity is an important performance index. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes: Amol Mangrulkar ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0276562.r001
Revision 1
21 Sep 2022 Author Response Reviewer #1: Thank you very much for the chance to review this manuscript. The authors extracted literature data on bone healing to train DL models for prediction modeling with regard to treatment success. I read the manuscript with great interest and think it provides an innovative way of processing the current data patterns in the literature to improve patient treatment outcomes. Although the available data points on this topic are small at the moment and the provided models’ capacities are insufficient for accurate predictions, I think this technique could be used in the future when more data is available in literature or other fields where sufficient data is already available. I recommend publishing this work as I believe it significantly contributes to the literature. I only have some minor comments which I recommend addressing before publication: Authors: We thank the reviewer for their positive and constructive comments, and we appreciate the reviewer's recognition of our purpose in providing this initial analysis as a framework for incorporating larger quantities of experimental data as they become available. Addressing the reviewer’s concerns has improved the clarity and thoroughness of our manuscript. 1) How was the literature search performed? The process of data collection (literature search) should be provided in more detail (e.g., search algorithm, which databases), as the results are based on them. A1.1) We searched the MEDLINE database using the PubMed search engine. We used specific orthobiologic factors as search terms. Specifics on the search are given in detail in the description of Step 1 in the Methods section of the revised manuscript. 2)P6 “The analysis indicates that combinations of BMP2, BMP7, and OG, perhaps including ES or BMAC, each at the high end of their ranges as reported in relevant studies, should outperform combinations that lack those components.” Which relevant studies? They should be cited after this statement. A1.2) This issue is confused because the sentence is misleading as written. It has been rewritten as: "The analysis indicates that combinations of BMP2 (15mg), BMP7 (5mg), and OG (150ug), each at the high end of their ranges as shown in the input/desired-output table, and perhaps including ES or BMAC (see Supplementary Text S1 and the Supplementary Spreadsheet), should outperform combinations that lack those components." The Supplementary Spreadsheet is an Excel spreadsheet that has every input/desired-output pair used in this analysis, with quantification in appropriate units as reported in the literature. Supplementary Text S1 provides details on how each input and desired output were measured, and also provides a complete list of the published articles from which all the input/desired-output data were drawn. References to the supplementary materials have been clarified throughout the revised text. 3) Please provide the hardware and software used for model building and training for reproducibility of methods provided A1.3) All computer programs were written in-house in MATLAB™ and run on Intel Core-i5 based Dell™ workstations. Algorithms were developed directly from their descriptions in the primary literature, as summarized and cited in Supplementary Text S2. The core set of programs and the entire dataset are available for free download at https://github.com/SigmoidNetmaker/deep-network-orthobiologics. Statements of these facts have been added to the revised version at the end of the first paragraph of Methods. 4) Usually, there is also a validation dataset used that the algorithms have not seen before, resulting in a train, test, and validation dataset. As the testing data will also be used in the hyperparameter tuning process (to evaluate the best parameters), it is recommended by available guidelines on the reporting of machine and deep learning algorithms to include this type of internal validation. However, I see that with the limited data points, this seems to be not feasible. I recommend discussing this point in 1-2 sentences in the discussion section. A1.4) We did divide the input/desired-output data into training and testing sets. In assessing the ability of each network type to generalize, we divided the set of 225 input/desired-output patterns into 175 training patterns and 50 testing patterns. We computed the root mean square (RMS) error over the 50 testing patterns after training on the 175 training patterns. We then retrained each network type 10 times, choosing a different training set and testing set at random each time. The best generalizing network type was the one having the lowest average RMS error over the 10 retrainings (see also Supplementary Texts S3 and S4). The full set of 225 input/desired-output patterns was used to train the best generalizing ANN type. This explanation has been added to the descriptions of Step 3, Step 4, and Step 5 in the Methods section of the revised manuscript. Reviewer #2: i. In abstract add more regarding the novel contribution of the manuscript. A2.1) We thank the reviewer for acknowledgement of the novelty of our contribution. Indeed, PubMed and GoogleScholar searches of the standard keywords “orthobiolologics” <AND> “machine learning” yield no matches (except that GoogleScholar finds our medRxiv preprint describing this work). Our use of machine learning in this context is unprecedented. This is now noted in the Abstract of the revised version. This and the reviewer’s other comments have helped us improve our manuscript. ii. Literature review is shallow in context of various modeling technics. A2.ii) The field of Machine Learning (ML) is vast. Still, it is only a small part of the statistical modeling field more generally. A deep review of ML, not to mention of statistical modeling more generally, is well beyond the scope of our article, whose purpose is to provide a framework for leveraging experimental data on bone healing to identify potentially new treatment combinations for use as adjuvants in orthopedic surgery. Our purpose is stated more clearly in the Introduction of the revised manuscript. iii. For Data Driven AI model specially Deep Learning Model needs large data, tyr to research and add more. A2.iii) Large datasets are indeed preferable when using machine learning to train ANN AIs, but a large dataset is simply not available in this case. For training our ANNs, we used all the experimental orthobiologics data we could find. We state this fact in the Methods section of the revised manuscript. Our intention is to provide a computational framework to which further data can be incorporated as they become available. We address these issues and limitations in the Introduction and Discussion. iv. Novelty of the work in terms of comparison to existing comparable models is missing,you can refere article for refrence,Automated skull damage detection from assembled skull model using computer vision and machine learning.Int. j. inf. tecnol. 13, 1785–1790 (2021). https://doi.org/10.1007/s41870-021-00752-5 A2.iv) This is the first time that ML and ANNs have been applied in the extraction of knowledge from a set of experimental observations on bone healing, and use of that knowledge to identify potentially effective combinations of biologic factors as adjuvants in orthopedic surgery. This is now mentioned in the Abstract and Introduction of the revised manuscript. As in many other areas of biomedicine, the application of ML and AI has grown rapidly in orthopedics because they often outperform older methods (eg statistical or biophysically based). Several extensive reviews specifically on ML in orthopedics have appeared recently. These are summarized and cited in the Introduction of the revised manuscript. The reviewed studies employed not only ANNs and ML but many other statistical modeling modalities as well; their description is outside the scope of our manuscript. Knowledge was extracted most often from radiological images (eg X-radiology, computed tomography, magnetic resonance, bone density, etc) but also from patient records, gate patterns, various sensors (eg force sensors, kinetic skeletal trackers, video, wearable device records, etc), and other kinds of measurements (eg biochemical, biomechanical, and bioelectrical). The knowledge extracted by the AI was then used for diagnosis of such disorders as osteo arthritis, ligament deficiency, cartilage lesion, rotator cuff pathology, spinal stenosis, scoliosis, and detection of weakness, age, fracture, tumor, and other bone abnormalities. The AI also can be used to predict surgical outcomes including survival, risk, complications, morbidity, mortality, treatment cost, duration as inpatient, and even patient satisfaction. In orthopedic surgery, AIs can be used to detect and identify surgical landmarks or targets such as specific bones, ligaments, and other skeletal components. None of the reviews, nor our own PubMed and Google Scholar searches, identifies anything like the framework we describe in our manuscript for using experimental data on bone healing to identify potentially effective combinations of biologic factors as adjuncts in orthopedic surgery. Our framework is based on standard, canonical ANNs and ML, which renders transparent the application of these forms of AI to orthobiological data. This is also stated in the Introduction of the revised manuscript. v.The running time of the proposed model is not counted in the experiment. Time complexity is an important performance index. A2.v) All ANNs were trained for 50,000 training iterations using backpropagation (and closely related algorithms), where each iteration constituted one input/desired-output presentation (see Supplementary Text S3). Algorithms were written in MATLAB™ and run on Intel Core-i5 based Dell™ workstations. Training for 50,000 iterations took about 15 minutes. Times for blocks of multiple trainings are based on the 15-minute time for 50,000 iterations. Approximate computing times are now provided in the descriptions of Step 3, Step 4, and Step 5 and in the initial paragraph of the Methods section of the revised manuscript. A fuller account of the time required for all of the ANN and ML optimizations we undertook is given in Text S3 in the revised version of the Supplement. Indeed, optimizing the main ML parameters for all 16 ANN types was a major undertaking and required about 1 month of computer time, not including autoencoder optimization and preliminary runs. The routines were run on dedicated lab machines so the long processing time did not interfere with other work in the lab. Attachments Attachment Submitted filename: OrthobioReviewResponse.docx https://doi.org/10.1371/journal.pone.0276562.r002
10 Oct 2022 Decision Letter - Kathiravan Srinivasan, Editor A Novel Application of Neural Networks to Identify Potentially Effective Combinations of Biologic Factors for Enhancement of Bone Fusion/Repair PONE-D-22-16681R1 Dear Dr. Anastasio, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Kathiravan Srinivasan Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors have sufficiently addressed all comments. The can be published in the current version from my side. Reviewer #2: The Author adress coment no1 which In abstract add more regarding the novel contribution of the manuscript and add the novelty of thier work in abstarct which i verifies and all others comments address in all respect , Now no futher comments my side. ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes: DR AMOL L. MANGRULKAR ******** https://doi.org/10.1371/journal.pone.0276562.r003
Formally Accepted
20 Oct 2022 Acceptance Letter - Kathiravan Srinivasan, Editor PONE-D-22-16681R1 A Novel Application of Neural Networks to Identify Potentially Effective Combinations of Biologic Factors for Enhancement of Bone Fusion/Repair Dear Dr. Anastasio: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Kathiravan Srinivasan Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0276562.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .