PONE-D-24-18044Predicting Diabetes Mellitus Metabolic Targets and Chronic Complications Transitions—Analysis Based on Natural Language Processing and Machine Learning ModelsPLOS ONE

Dear Dr. Rodríguez-Lesmes,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Sep 27 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Ozra Tabatabaei-Malazy

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, we expect all author-generated code to be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse.

3. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match. 

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

4. Thank you for stating the following financial disclosure: 

"This study was supported by the Ministry of Science, Technology and Innovation (Minciencias) of Colombia. Grant Number: 138-2021, Universidad del Rosario, Fundación Universitaria Sanitas, and Universidad EIA in Colombia. Andrés García and Paul Rodríguez acknowledge support by the Ministry of Science, Technology and Innovation (Minciencias) of Colombia. Grant Number: 138-2021"

Please state what role the funders took in the study.  If the funders had no role, please state: ""The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."" 

If this statement is not correct you must amend it as needed. 

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

5. Thank you for stating the following in your Competing Interests section:  

"None"

Please complete your Competing Interests on the online submission form to state any Competing Interests. If you have no competing interests, please state ""The authors have declared that no competing interests exist."", as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now 

 This information should be included in your cover letter; we will change the online submission form on your behalf.

6. When completing the data availability statement of the submission form, you indicated that you will make your data available on acceptance. We strongly recommend all authors decide on a data sharing plan before acceptance, as the process can be lengthy and hold up publication timelines. Please note that, though access restrictions are acceptable now, your entire data will need to be made freely accessible if your manuscript is accepted for publication. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If you are unable to adhere to our open data policy, please kindly revise your statement to explain your reasoning and we will seek the editor's input on an exemption. Please be assured that, once you have provided your new statement, the assessment of your exemption will not hold up the peer review process.

7. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: Yes

Reviewer #3: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Summary:

In this study, the authors use natural language processing (NLP) and machine learning (ML) to predict diabetes progression from structured and unstructured electronic health records (EHRs).

Comments:

- The authors state that patients with a confirmed diagnosis of diabetes were included. Is confirmed diagnosis referred to ICD codes or specific diagnosis definitions used by the EHR system? Were only patients with type 2 diabetes included or also with type 1 diabetes?

- Why did the authors choose not to include neuropathy, peripheral vascular disease, and cardiovascular disease as complications?

- How were the complications defined? For chronic kidney disease, were also estimated glomerular filtration rates considered?

- The authors state that they used cross-validation to find the best performing model. However, they did not provide any information on whether cross-validation was just used for hyperparameter tuning or whether a nested cross-validation scheme was used to also get unbiased estimates of the models’ performance. Further, no information on the number of folds was provided.

- The results suggest that the ML models did not learn anything during training. For a binary classifier to achieve an F1-score of 0.95 and an AUC of 0.5, the dataset has to be very imbalanced (as is the case in the paper) and the predicted probabilities must be all very close to zero or one (depending whether the majority class labels are zero or one). The authors should carefully check their ML pipeline again.

- Mean HbA1c levels are not reported in Table 1.

- I would expect the authors to provide the full list of covariates that were used for the ML models.

- I would suggest removing Figure 1. It is clear from the text what is meant and the Figure does not look professional enough for a scientific publication.

- I am not an expert in NLP, thus I cannot assess whether the bag-of-words method is a reasonable choice for the task at hand.

Reviewer #2: This is a generally solid study in terms of both methodology and manuscript quality. The concept is interesting and somewhat unique within the broader landscape of published research. However, there are some areas where the manuscript could be improved, particularly by addressing certain missing details in the methodology:

Methods:

Population Description:

The description of the population lacks specifics regarding inclusion and exclusion criteria. For example, the phrase "patients with a confirmed diagnosis of DM" is vague. It would be helpful to specify the diagnostic criteria used for DM and clarify whether certain patient groups (e.g., those with a recent diagnosis, pregnancy, etc.) were excluded from the study.

NLP Methodology:

The Natural Language Processing (NLP) techniques described require more detailed explanation regarding the algorithms and parameters used. For instance, while you mention the creation of a custom medical dictionary, it is unclear how this was integrated into the NLP pipeline (e.g., which specific NLP models or tools, such as SpaCy, NLTK, etc., were utilized). Additionally, provide clarification on how the accuracy of the labels generated by the NLP methods was validated.

Model Performance Metrics:

The results section mentions F1 scores and accuracy for different models but lacks discussion on how these metrics compare across models and the reasons certain models outperformed others. Including a table summarizing the performance metrics for each model, along with a brief discussion of the strengths and weaknesses of each approach, would enhance the clarity of the findings.

ML Model Selection:

The manuscript references several machine learning algorithms (KNN, LR, DT, RF, NN, Boosting) but does not provide a rationale for selecting these specific models. Offering a brief justification for each model's selection based on the nature of the data and the research problem would strengthen the rigor of the methodology section. Additionally, more details on the hyperparameter optimization process and how it was tailored to each model would be beneficial.

Handling of Missing Data:

There is no discussion of how missing data were addressed, particularly in the EHR records. It would be helpful to specify the approach taken (e.g., imputation methods, exclusion of incomplete cases) and discuss the potential impact of missing data on the study’s results.

Discussion:

Clinical Implications:

The conclusion discusses the potential impact of the study but could be more specific. For instance, you mention the possibility of "improved health outcomes" without specifying which outcomes might be most affected. Consider providing concrete examples of how the findings could influence clinical practice, such as specific interventions that could be implemented This is a generally solid study in terms of both methodology and manuscript quality. The concept is interesting and somewhat unique within the broader landscape of published research. However, there are some areas where the manuscript could be improved, particularly by addressing certain missing details in the methodology:

Methods:

Population Description:

The description of the population lacks specifics regarding inclusion and exclusion criteria. For example, the phrase "patients with a confirmed diagnosis of DM" is vague. It would be helpful to specify the diagnostic criteria used for DM and clarify whether certain patient groups (e.g., those with a recent diagnosis, pregnancy, etc.) were excluded from the study.

NLP Methodology:

The Natural Language Processing (NLP) techniques described require more detailed explanation regarding the algorithms and parameters used. For instance, while you mention the creation of a custom medical dictionary, it is unclear how this was integrated into the NLP pipeline (e.g., which specific NLP models or tools, such as SpaCy, NLTK, etc., were utilized). Additionally, provide clarification on how the accuracy of the labels generated by the NLP methods was validated.

Model Performance Metrics:

The results section mentions F1 scores and accuracy for different models but lacks discussion on how these metrics compare across models and the reasons certain models outperformed others. Including a table summarizing the performance metrics for each model, along with a brief discussion of the strengths and weaknesses of each approach, would enhance the clarity of the findings.

ML Model Selection:

The manuscript references several machine learning algorithms (KNN, LR, DT, RF, NN, Boosting) but does not provide a rationale for selecting these specific models. Offering a brief justification for each model's selection based on the nature of the data and the research problem would strengthen the rigor of the methodology section. Additionally, more details on the hyperparameter optimization process and how it was tailored to each model would be beneficial.

Handling of Missing Data:

There is no discussion of how missing data were addressed, particularly in the EHR records. It would be helpful to specify the approach taken (e.g., imputation methods, exclusion of incomplete cases) and discuss the potential impact of missing data on the study’s results.

Discussion:

Clinical Implications:

The conclusion discusses the potential impact of the study but could be more specific. For instance, you mention the possibility of "improved health outcomes" without specifying which outcomes might be most affected. Consider providing concrete examples of how the findings could influence clinical practice, such as specific interventions that could be implemented based on the model's predictions.

Limitations:

The limitations section is somewhat underdeveloped. While you mention a short follow-up period and the lack of data on hospitalizations or mortality, other potential limitations, such as selection bias, the accuracy of NLP-generated labels, and the generalizability of the findings to other populations, are not addressed. Expanding on these aspects will provide a more comprehensive assessment of the study's limitations.

General:

Consistency in Terminology:

Ensure consistency in terminology throughout the manuscript. For example, the terms "out of targets" and "outside goals" are used interchangeably in different sections. Standardizing the language will help avoid confusion and improve readability.

Conclusion:

Incorporating more detailed explanations of the methodology and discussing the practical implications of the study in day-to-day clinical practice could significantly enhance the value of this already valuable study.

on the model's predictions.

Limitations:

The limitations section is somewhat underdeveloped. While you mention a short follow-up period and the lack of data on hospitalizations or mortality, other potential limitations, such as selection bias, the accuracy of NLP-generated labels, and the generalizability of the findings to other populations, are not addressed. Expanding on these aspects will provide a more comprehensive assessment of the study's limitations.

General:

Consistency in Terminology:

Ensure consistency in terminology throughout the manuscript. For example, the terms "out of targets" and "outside goals" are used interchangeably in different sections. Standardizing the language will help avoid confusion and improve readability.

Conclusion:

Incorporating more detailed explanations of the methodology and discussing the practical implications of the study in day-to-day clinical practice could significantly enhance the value of this already valuable study.

Reviewer #3: Dear Authors,

I hope this message finds you well.

We have reviewed your paper titled “Predicting Diabetes Mellitus Metabolic Targets and Chronic Complications Transitions—Analysis Based on Natural Language Processing and Machine Learning Models.” While your work provides a sophisticated insight into the application of hybrid machine learning methods in disease diagnosis, there are several major considerations that need to be addressed before further processing. Please review and address the following points:

Journal Style and Format: At first glance, the current context of your paper does not align with the typical structure of a journal article. It reads more like a summary report with limited data. Please ensure that your manuscript adheres to the journal’s style guidelines.

Literature Review: The literature review is insufficient. Typically, a robust review should include around 20 related articles focusing on the application of ML and NLP in disease diagnosis. Your review lacks depth and does not cover trends in disease complications. Please review at least 10 journal articles in your field that utilize machine learning and provide a comparative table of their results.

NLP Methods: You mention the implementation of NLP but do not specify which methods were used. There are three main NLP methods available. Please clarify which ones were implemented in your study.

NLP Metrics: You mentioned using two approaches, simple search and BoW, and ultimately selecting BoW. However, you did not report any clear metrics. If these metrics are in the appendix, please note that the appendix is not the appropriate place for them. Include clear metrics in the main text.

Machine Learning Methods: In the materials and methods section, you refer to both regression and classification methods, but your study focuses on classification. This makes the mention of regression methods unnecessary. It appears you implemented only neural networks but reported other methods as well. Please clarify which methods were implemented in sufficient detail.

Results and Discussion: For NLP, it is necessary to provide a table of metrics to compare all implemented groups, including data size (optional) and accuracy. Is this information in Table 2? Please clarify.

Classification Results: Clarify which classification method performed best and provide a consistent conclusion about the potential of your study, reducing unnecessary parts.

Figures: Please split Figure 2 into three separate figures, as it contains a lot of information. Combining them into one figure may reduce their quality.

Best regards

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: No

Reviewer #2: Yes:  Sepehr Khosravi

Reviewer #3: Yes:  Dr. Parviz Narimani

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

PONE-D-24-18044R1Predicting Diabetes Mellitus Metabolic Goals and Chronic Complications Transitions—Analysis Based on Natural Language Processing and Machine Learning Models.PLOS ONE

Dear Dr. Rodríguez-Lesmes,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

Please submit your revised manuscript by Dec 06 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Ozra Tabatabaei-Malazy

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: (No Response)

Reviewer #4: Dear Authors,Thanks for implementation of previous comments on the paper;moreover, with the minor revision on these areas, the paper would be acceptabe:

Detailed NLP Preprocessing Steps:

Clarity and Reproducibility: The manuscript would benefit from a more detailed description of the NLP preprocessing steps. For instance, specifying techniques used for text normalization (e.g., stemming, lemmatization), handling of stopwords, and dealing with medical abbreviations and jargon could improve clarity. Explaining these choices can help other researchers replicate your findings and understand the basis of your NLP pipeline’s effectiveness.

Error Analysis: Discuss any challenges faced during the text preprocessing, such as misclassifications or unusual patterns in the data. Addressing how these were handled or mitigated would provide insights into the robustness of the NLP methodology.

Justification for ML Algorithm Selection:

Algorithm Suitability: While the manuscript employs a variety of ML algorithms, there is little discussion on why these specific algorithms were chosen over others. For example, detailing the reasons for selecting neural networks or decision trees based on their suitability for the dataset characteristics (e.g., size, feature type, non-linearity) would enrich the analysis.

Performance Comparison: Provide a comparative analysis of the ML algorithms used. This could include discussions on each algorithm's performance with respect to handling imbalanced data, computational efficiency, and ease of interpretation. Such comparisons could be instrumental in justifying the choice of one model over another.

Model Validation Techniques:

Validation Strategy: The manuscript mentions the use of cross-validation but does not specify the type (e.g., k-fold, leave-one-out). Detailing the choice of cross-validation method, the number of folds used, and how the data was split can affect model evaluation significantly, especially in medical data where patient independence is crucial.

Handling of Imbalanced Data: Medical datasets often have class imbalance issues, particularly in disease prediction. Discussing strategies employed to handle imbalance, such as synthetic minority oversampling technique (SMOTE) or adjusting class weights in model training, would be valuable.

Hyperparameter Optimization:

Optimization Details: Expanding on the hyperparameter optimization process would strengthen the methodological rigor. For instance, detailing the parameters explored, the range of values tested, and the optimization algorithm (e.g., grid search, random search, Bayesian optimization) could provide deeper insights.

Impact on Model Performance: Discuss how hyperparameter tuning affected the performance of each model. This could include changes in model accuracy, sensitivity, specificity, and computational demands. Such details could help justify the final model configuration chosen for the study.

Performance Metrics Explanation:

Metric Selection Rationale: While accuracy, F1 score, and AUC are standard, discussing why these were chosen and their relevance to the study’s clinical implications would be insightful. For instance, in a medical context, the cost of false negatives might be more critical than false positives.

Thresholds and Trade-offs: Elaborate on any thresholds used for classification decisions (e.g., cut-off probabilities for classifying a patient’s disease stage). Discussing the trade-offs between sensitivity and specificity in choosing these thresholds, especially how they might impact clinical decisions, would be extremely beneficial.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #2: Yes:  Sepehr Khosravi

Reviewer #4: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

PONE-D-24-18044R2Predicting Diabetes Mellitus Metabolic Goals and Chronic Complications Transitions—Analysis Based on Natural Language Processing and Machine Learning Models.PLOS ONE

Dear Dr. Rodríguez-Lesmes,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

=========================================================

Please submit your revised manuscript by Feb 03 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Ozra Tabatabaei-Malazy

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #4: 1. Employ Additional Visual Representations:

Include one or two essential graphs illustrating machine learning performance, such as: A confusion matrix for the optimal model to illustrate prediction accuracy for each class. A bar chart illustrating the F1-scores of various models to facilitate comparisons for readers.

2. Emphasize Feature Significance: Incorporate a basic feature importance plot for a primary machine learning model (e.g., Random Forest or Logistic Regression) to illustrate which factors exerted the most significant influence on predictions.

3. Concisely Elucidate Model Selections: Incorporate a brief paragraph into the main text (citing Appendix 3) that elucidates the rationale for the selection of specific machine learning models, highlighting the interpretability of Logistic Regression and the robustness of Random Forest.

4. Elucidate Data Preprocessing:

Concise preprocessing measures, including as oversampling and feature selection, were implemented to ensure the models adeptly managed unbalanced input.

5. Concentrate on Principal Outcomes: Identify a significant discovery from the machine learning models, such as the primary predictor for complications or metabolic objectives, and its correlation with clinical observations.

6. Examine Constraints Gently:

One restriction is the problem of imbalanced data or dependence on the quality of EHR text, which was partially mitigated using approaches such as NLP and oversampling.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #4: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

PONE-D-24-18044R3Predicting Diabetes Mellitus Metabolic Goals and Chronic Complications Transitions—Analysis Based on Natural Language Processing and Machine Learning Models.PLOS ONE

Dear Dr. Rodríguez-Lesmes,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

Please submit your revised manuscript by Apr 09 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Ozra Tabatabaei-Malazy

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #4: It is evident that all comments have been addressed, and the paper exhibits a profound and proficient writing style. However, it is recommended that additional tables from the support document, entitled "S3 ML Methods Detail," such as Tables A3.1 and 3.2, be incorporated into the original body of the paper to provide further substantiation and clarity.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #4: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

Predicting Diabetes Mellitus Metabolic Goals and Chronic Complications Transitions—Analysis Based on Natural Language Processing and Machine Learning Models.

PONE-D-24-18044R4

Dear Dr. Rodríguez-Lesmes,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager®  and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Ozra Tabatabaei-Malazy

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments: