PCSY-D-25-00001

A Single-Graph Visualization to Reveal Hidden Explainability Patterns of SHAP Feature Interactions in Machine Learning for Biomedical Issues

PLOS Complex Systems

Dear Dr. Monsarrat,

Thank you for submitting your manuscript to PLOS Complex Systems. After careful consideration, we feel that it has merit but does not fully meet PLOS Complex Systems's publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript within 60 days May 26 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at complexsystems@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pcsy/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

* A rebuttal letter that responds to each point raised by the editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. This file does not need to include responses to any formatting updates and technical items listed in the 'Journal Requirements' section below.

* A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

* An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, competing interests statement, or data availability statement, please make these updates within the submission form at the time of resubmission. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

We look forward to receiving your revised manuscript.

Kind regards,

Juan Gonzalo Barajas-Ramirez

Academic Editor

PLOS Complex Systems

Juan Gonzalo Barajas-Ramirez

Academic Editor

PLOS Complex Systems

Hocine Cherifi

Editor-in-Chief

PLOS Complex Systems

Journal Requirements:

1. We have amended your Competing Interest statement to comply with journal style. We kindly ask that you double check the statement and let us know if anything is incorrect.

2. We ask that a manuscript source file is provided at Revision. Please upload your manuscript file as a .doc, .docx, .rtf or .tex.

3. Please provide an Author Summary. This should appear in your manuscript between the Abstract (if applicable) and the Introduction, and should be 150–200 words long. The aim should be to make your findings accessible to a wide audience that includes both scientists and non-scientists. Sample summaries can be found on our website under Submission Guidelines:

https://journals.plos.org/complexsystems/s/submission-guidelines#loc-parts-of-a-submission

4. Please upload a copy of Figures 1 and 2 which you refer to in your text on pages 6 and 11. Or, if the figure is no longer to be included as part of the submission please remove all reference to it within the text.

6. For studies involving third-party data, we encourage authors to share any data specific to their analyses that they can legally distribute. PLOS recognizes, however, that authors may be using third-party data they do not have the rights to share. When third-party data cannot be publicly shared, authors must provide all information necessary for interested researchers to apply to gain access to the data. (https://journals.plos.org/plosone/s/data-availability#loc-acceptable-data-access-restrictions)

For any third-party data that the authors cannot legally distribute, they should include the following information in their Data Availability Statement upon submission:

a) A description of the data set and the third-party source

b) If applicable, verification of permission to use the data set

c) Confirmation of whether the authors received any special privileges in accessing the data that other researchers would not have

d) All necessary contact information others would need to apply to gain access to the data

Additional Editor Comments (if provided):

AE: The reviewers are divided on the recommendation of the manuscript. The main concerns are the clarity and tractability in a robust way to avoid questions about its scientific rigor. The authors are encourage to prepare a revised version of the manuscript where each point raised by the reviewers.

Reviewer 1.

The manuscript proposes a novel single-graph visualization method to uncover hidden interaction patterns in SHAP-based explanations for biomedical ML models. The topic is highly relevant, particularly for improving transparency in healthcare applications. The proposed method is a promising contribution to interpretable ML in biomedicine. However, revisions addressing the above points—particularly methodological transparency, experimental breadth, and discussion of limitations—are essential to ensure scientific rigor and reproducibility. With these improvements, the manuscript will significantly advance the field of explainable AI for healthcare. However, several critical revisions are needed to strengthen methodological rigor, experimental validation, and clarity.

1.The literature review lacks a systematic comparison with existing visualization tools (e.g., SHAP dependence plots, interaction heatmaps). This weakens the justification for the proposed method’s novelty.

2. The rationale for selecting the interaction threshold (0.22) is unexplained.

3. No discussion on handling high-dimensional interaction tensors (e.g., dimensionality reduction or filtering strategies).

4. Justification for the Spearman correlation threshold (-0.3 to 0.3) is absent.

5. Validation is limited to a single dataset (NHANES), raising concerns about generalizability.

6. Model performance metrics (e.g., RMSE, R² for the XGBoost age prediction task) are omitted, making it unclear whether interaction analysis depends on a high-quality predictive model.

7. Statistical significance of reported correlations (e.g., p-values for Spearman coefficients) is not addressed.

8. The method assumes monotonic relationships, but non-linear or higher-order interactions are common in biomedical systems.

9. Inconsistent terminology (e.g., “Glycohemoglobin” vs. “Glycated Hemoglobin”).

10. Figure annotations lack clarity (e.g., color gradients in Fig. 1B/C are undefined).

11. Reference formatting is inconsistent (e.g., missing volume numbers, page ranges) , suggest adding references.

Reviewer 2.

This manuscript presents a novel single-graph visualization method for SHAP-based feature interactions in biomedical machine learning models. While the topic is important, further validation and refinement are needed to meet the standards of PLOS Complex Systems.

1. The paper lacks a comprehensive comparison with existing SHAP visualization techniques. Clarifying how this method enhances interpretability and usability over existing SHAP visualization techniques would better highlight its unique contribution.

2. The study was applied solely to the NHANES dataset. Evaluating the method on additional datasets, especially those with varied feature interactions, would improve its robustness and generalizability.

3. Figure 2 appears to omit certain pairwise interactions. For instance, the pattern of Cholesterol (red node) → ALT (blue node) as a positive interaction and ALT (blue node) → Cholesterol (red node) as a negative interaction is missing, even though their main effects indicate a potential relationship.

Reviewer 3

I have reviewed the manuscript "A Single-Graph Visualization to Reveal Hidden Explainability Patterns of SHAP Feature Interactions in Machine Learning for Biomedical Issues" by Furger and coauthors, submitted for publication in PLOS Complex Systems.

This manuscript introduces a straightforward graph visualization for explainability patterns in machine learning methods, particularly focusing on feature interactions - a crucial aspect that has not received sufficient attention from the scientific community. The authors utilize the SHAP TreeExplainer output to construct a graph wherein nodes represent model features, and connections indicate whether one feature amplifies or attenuates the main effect of another. The method is effectively demonstrated using the NHANES database in the task of predicting chronological age based on numerous biological variables.

The manuscript is clear, well-structured, and easy to read. I believe the approach represents a valuable contribution to addressing the critical need for explainability in complex machine learning models and will likely capture the interest of various research communities. However, I have a few minor suggestions for the authors to consider prior to publication:

- It would be beneficial to cite the original articles when referencing tools such as LIME, SHAP, XGBoost, and others.

- Mentioning related methods, such as Accumulated Local Effects (ALE) and Friedman's H-statistic, which also allow visualization of feature interactions, could further contextualize this work and highlight its unique advantages.

- A potential improvement for the graph visualization in Figure 2 would be explicitly representing the interaction intensity, perhaps by varying the edge widths according to their strength.

- Although the tool is currently available on GitHub, I recommend that the authors also distribute it via the Python Package Index (PyPI) to enhance its visibility and accessibility.

[Note: HTML markup is below. Please do not edit.]

Reviewers' Comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does this manuscript meet PLOS Complex Systems’s publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS Complex Systems does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The manuscript proposes a novel single-graph visualization method to uncover hidden interaction patterns in SHAP-based explanations for biomedical ML models. The topic is highly relevant, particularly for improving transparency in healthcare applications. The proposed method is a promising contribution to interpretable ML in biomedicine. However, revisions addressing the above points—particularly methodological transparency, experimental breadth, and discussion of limitations—are essential to ensure scientific rigor and reproducibility. With these improvements, the manuscript will significantly advance the field of explainable AI for healthcare. However, several critical revisions are needed to strengthen methodological rigor, experimental validation, and clarity.

1.The literature review lacks a systematic comparison with existing visualization tools (e.g., SHAP dependence plots, interaction heatmaps). This weakens the justification for the proposed method’s novelty.

2. The rationale for selecting the interaction threshold (0.22) is unexplained.

3. No discussion on handling high-dimensional interaction tensors (e.g., dimensionality reduction or filtering strategies).

4. Justification for the Spearman correlation threshold (-0.3 to 0.3) is absent.

5. Validation is limited to a single dataset (NHANES), raising concerns about generalizability.

6. Model performance metrics (e.g., RMSE, R² for the XGBoost age prediction task) are omitted, making it unclear whether interaction analysis depends on a high-quality predictive model.

7. Statistical significance of reported correlations (e.g., p-values for Spearman coefficients) is not addressed.

8. The method assumes monotonic relationships, but non-linear or higher-order interactions are common in biomedical systems.

9. Inconsistent terminology (e.g., “Glycohemoglobin” vs. “Glycated Hemoglobin”).

10. Figure annotations lack clarity (e.g., color gradients in Fig. 1B/C are undefined).

11. Reference formatting is inconsistent (e.g., missing volume numbers, page ranges) , suggest adding references.

Reviewer #2: This manuscript presents a novel single-graph visualization method for SHAP-based feature interactions in biomedical machine learning models. While the topic is important, further validation and refinement are needed to meet the standards of PLOS Complex Systems.

1. The paper lacks a comprehensive comparison with existing SHAP visualization techniques. Clarifying how this method enhances interpretability and usability over existing SHAP visualization techniques would better highlight its unique contribution.

2. The study was applied solely to the NHANES dataset. Evaluating the method on additional datasets, especially those with varied feature interactions, would improve its robustness and generalizability.

3. Figure 2 appears to omit certain pairwise interactions. For instance, the pattern of Cholesterol (red node) → ALT (blue node) as a positive interaction and ALT (blue node) → Cholesterol (red node) as a negative interaction is missing, even though their main effects indicate a potential relationship.

Reviewer #3: I have reviewed the manuscript "A Single-Graph Visualization to Reveal Hidden Explainability Patterns of SHAP Feature Interactions in Machine Learning for Biomedical Issues" by Furger and coauthors, submitted for publication in PLOS Complex Systems.

This manuscript introduces a straightforward graph visualization for explainability patterns in machine learning methods, particularly focusing on feature interactions - a crucial aspect that has not received sufficient attention from the scientific community. The authors utilize the SHAP TreeExplainer output to construct a graph wherein nodes represent model features, and connections indicate whether one feature amplifies or attenuates the main effect of another. The method is effectively demonstrated using the NHANES database in the task of predicting chronological age based on numerous biological variables.

The manuscript is clear, well-structured, and easy to read. I believe the approach represents a valuable contribution to addressing the critical need for explainability in complex machine learning models and will likely capture the interest of various research communities. However, I have a few minor suggestions for the authors to consider prior to publication:

- It would be beneficial to cite the original articles when referencing tools such as LIME, SHAP, XGBoost, and others.

- Mentioning related methods, such as Accumulated Local Effects (ALE) and Friedman's H-statistic, which also allow visualization of feature interactions, could further contextualize this work and highlight its unique advantages.

- A potential improvement for the graph visualization in Figure 2 would be explicitly representing the interaction intensity, perhaps by varying the edge widths according to their strength.

- Although the tool is currently available on GitHub, I recommend that the authors also distribute it via the Python Package Index (PyPI) to enhance its visibility and accessibility.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.

For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Tianlong Yuan

Reviewer #2: No

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

Figure resubmission:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. If there are other versions of figure files still present in your submission file inventory at resubmission, please replace them with the PACE-processed versions.

Reproducibility:

To enhance the reproducibility of your results, we recommend that authors of applicable studies deposit laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

PCSY-D-25-00001R1

A Single-Graph Visualization to Reveal Hidden Explainability Patterns of SHAP Feature Interactions in Machine Learning for Biomedical Issues

PLOS Complex Systems

Dear Dr. Monsarrat,

Thank you for submitting your manuscript to PLOS Complex Systems. After careful consideration, we feel that it has merit but does not fully meet PLOS Complex Systems's publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript within 30 days Jul 04 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at complexsystems@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pcsy/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

* A rebuttal letter that responds to each point raised by the editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. This file does not need to include responses to any formatting updates and technical items listed in the 'Journal Requirements' section below.

* A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

* An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, competing interests statement, or data availability statement, please make these updates within the submission form at the time of resubmission. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

We look forward to receiving your revised manuscript.

Kind regards,

Juan Gonzalo Barajas-Ramirez

Academic Editor

PLOS Complex Systems

Juan Gonzalo Barajas-Ramirez

Academic Editor

PLOS Complex Systems

Hocine Cherifi

Editor-in-Chief

PLOS Complex Systems

Additional Editor Comments (if provided):

The reviewers find the revised version of the manuscript much improved with Reviewer 2 requiring additional corrections mainly in the format of figures. Therefore the authors are invited to prepared a revised version of the manuscript addressing fully the comments provided by the reviewers.

[Note: HTML markup is below. Please do not edit.]

Reviewers' Comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Does this manuscript meet PLOS Complex Systems's publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS Complex Systems does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: 1) The y-axis scale in Figure 1B and Figure 1C seems inconsistent. It would be clearer if the same y-axis scale could be used to facilitate direct comparison of the raw interaction values.

2) The y-axis values in Figures 1C and 1D differ by approximately a factor of two. Given that they both derive from the same adjusted SHAP interaction data, it seems they should have the same y-axis data. Please check and clarify the reason for this discrepancy.

3) In Figures 1 and 3, the axis label font sizes are too small, making them difficult to read. Please consider adjusting the font size to enhance readability.

4) The plots in Figure 1D, 3B, and 3C do not show p-values for the reported Spearman correlations. Including p-values would add further statistical rigor.

5) In Figure 3A, the color scale is not quantitatively labeled. Please provide numerical scale values to ensure clarity.

6) In Section 2 (Material and Methods), while the references for SHAP values are cited, it would be valuable to clearly define SHAP values and the interaction tensor I(i, j, k), including the corresponding formula for clarity.

Reviewer #3: I thank the authors for considering all my comments in this revised version.

The authors' replies and changes in the manuscript adequately address all the

minor comments I raised in the first report. Thus, I can only congratulate the

authors on their work, which I recommend for publication in its present form.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.

For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

Figure resubmission:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. If there are other versions of figure files still present in your submission file inventory at resubmission, please replace them with the PACE-processed versions.

Reproducibility:

To enhance the reproducibility of your results, we recommend that authors of applicable studies deposit laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

A Single-Graph Visualization to Reveal Hidden Explainability Patterns of SHAP Feature Interactions in Machine Learning for Biomedical Issues

PCSY-D-25-00001R2

Dear Pr Monsarrat,

We are pleased to inform you that your manuscript 'A Single-Graph Visualization to Reveal Hidden Explainability Patterns of SHAP Feature Interactions in Machine Learning for Biomedical Issues' has been provisionally accepted for publication in PLOS Complex Systems.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow-up email from a member of our team. 

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they'll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact complexsystems@plos.org.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Complex Systems.

Best regards,

Juan Gonzalo Barajas-Ramirez

Academic Editor

PLOS Complex Systems

Hocine Cherifi

Editor-in-Chief

PLOS Complex Systems

***********************************************************

The reviewers have completed their evaluation of the second revised version of the manuscript and are now recommending acceptance of this version.

Reviewer Comments (if any, and for reference):

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Does this manuscript meet PLOS Complex Systems's publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS Complex Systems does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: All comments have been addressed. The authors have adequately responded to the reviewers’ concerns and improved the manuscript’s clarity and presentation. I have no further concerns and recommend the revised version for publication.

Reviewer #3: As before, I have no further comments and recommend the publication of this

work in its present form.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.

For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

Reviewer #3: No

**********