PONE-D-23-33548Validity of the information on the basic cause of death: an analysis in Colombia during the COVID-19 pandemic in 2021PLOS ONE

Dear Dr. Berrio-Parra,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Your article addresses a crucial issue concerning the disparities between causes of death (CoD) in death certificates and other sources including clinical records, which form the basis for much epidemiological and demographical data, influencing healthcare policy and decision-making in Colombia and beyond. While the study's objective is clear, the manuscript lacks sufficient detail to meet PLOS ONE's publication criteria. I am outlining the main issues that must be addressed for the manuscript to be considered suitable for publication in our journal.

1. Clear definition of PICO: Please restate the PICO structure research question clearly, focusing on the "reference gold standard CoD source" and the "original CoD source." Provide detailed selection criteria, sampling methods, and information about the study population. Revise all calculations thoroughly to ensure accurate comparisons, and report the main outcomes according to the planned methods to prevent spin bias.

2. Reproducible methods and research integrity compliance: To enhance reproducibility, provide full details about the CoD adjudicating committee and methods for determining causes of death.

3. Conclusions supported by data: Ensure that the discussion section addresses the implications of your research findings. Most importantly, suggest possible solutions that could improve accurate CoD adjudication, serving as a foundation for better healthcare policy and decision-making.

4. Reporting guidelines and community standards: Comply with the IMRaD structure for scientific articles, including quality content items for the abstract section. It is highly recommended to follow the abstract and full-text STROBE statements for observational studies.

5. Article is written in an intelligible fashion and is in standard English: Perform a thorough English copyediting and coherence revision to ensure clear understanding of the research. Present manuscript sections, tables, and figures in an organized and appropriately classified manner.

Please submit your revised manuscript by Jan 15 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Andres Mauricio Acevedo-Melo, M.D.

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf.

2. Note from Emily Chenette, Editor in Chief of PLOS ONE, and Iain Hrynaszkiewicz, Director of Open Research Solutions at PLOS: Did you know that depositing data in a repository is associated with up to a 25% citation advantage (https://doi.org/10.1371/journal.pone.0230416)? If you’ve not already done so, consider depositing your raw data in a repository to ensure your work is read, appreciated and cited by the largest possible audience. You’ll also earn an Accessible Data icon on your published paper if you deposit your data in any participating repository (https://plos.org/open-science/open-data/#accessible-data).

3. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

""Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

4. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well. 

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: No

Reviewer #3: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The manuscript presents several methodological concerns, including unclear processes for informed consent when accessing clinical records, potential legal ramifications due to the sensitive nature of death diagnoses, undisclosed qualifications of the data reviewers, and doubts about data anonymization and interview consent.

Reviewer #2: Although the document presents a well-known statistical technique, such as Kappa, it is nothing more than an academic exercise that in real life does not advance the possibility of correcting poorly coded deaths for better decision making.

Reviewer #3: Dear authors, the paper is interesting and analyzes a crucial problem. Nevertheless, I suggest you an additional effort to better clarify some aspects of the methods and to improve the consistency of data analysis.

General remarks

1. The declared objective of the paper (line 82-85) is to compare the ‘original certificate’ with a more accurate information, so the “standard certificate” should be always used as the gold standard in the statistical analysis. Nevertheless, some indicators are calculated taking the original certificate as a reference. For instance, given a cause of death J, “false negatives” are certificates classified as J with the gold standard and classified differently by the “original certificate” (which is under evaluation). “False positives” should be certificates in which the cause is other than J for the gold standard J for the “original certificate”. For example, for ICD10 chapter I, the false positives are 9 (from table 1), and the percentage of false positive 1.20. In the same way, “false negatives” are 8 and the percentage of false negative is 8/23*100= (34.8). (see table in the attached file). Nevertheless, in table 3 the percentages of false positives and false negatives are calculated using the original certificate as the gold standard. I suggest the authors to calculate the indicators using the standard certificate as the reference.

The indicator presented in line 114 seems to be calculated consistently with the objectives (standard certificate as gold standard) but the results presented in the last column of table 3 seem to have inverted signs.

In the discussion, line 207, the authors report that there is an underestimation of (for instance) circulatory system disease, but in reality, the original certificate overestimates these causes compared to the gold standard statistics.

What the authors think about this? It would be better to calculate the indicators using the standard certificate as gold standard.

2. I suggest to clarify the methods (Ln 95-106):

a. The trained health professionals, fill-in a new death form (part 1 and part 2 of the standard death certificate) or attribute a single cause-of-death?

b. Does the underlying cause codes for the original certificate correspond to those of official statistics for Colombia?

c. Is there a hierarchy of the medical documentation used for attributing the cause of death for the standard certificate? For deaths with more than one type of medical document (necropsy, clinical records, interview to relatives etc) only one is used for attributing the cause of death (as it appears from table 1)? In which order?

Additional specific topics:

1. I suggest to use the standard term “underlying cause” instead of “basic cause”.

2. When ICD chapters are mentioned, I would suggest to use also the chapter number, that would allow to find the results more easily in the tables.

3. Acronyms should be explained only once, the first time that are used (examples in line 28, CoD acronym described in the line above).

4. Ln 32-33. The sentence is not clear, what the author mean by “chart review” and “those who knew the medical history”? The terms are clear in the paper but they should be clear also in the abstract. Should it be “medical documentation” and “interviews to relatives”? Ln 33: the certificate considered “standard” is the one based on the medical documentation and interviews?

5. Ln 116-117, which variable and which points are plotted in the Bland-Altman diagram?

6. Ln 124-130 and table 1. Is the sample analyzed (776 cases) representative of the total deaths in Colombia (in terms of age and sex)? A sentence regarding this aspect and on the possible distortion due to the loss of some cases from the desired sample would be useful.

7. Ln 157-164. Confidence interval for Kappa should be used for comments and only robust values should be highlighted. For instance, results found for chapter “pregnancy childbirth and puerperium” are based only on two cases: it is not possible to evaluate false positives and Kappa is misleading. This should be taken into account.

8. Ln 167: The term “level of occurrence” is not clear. The table represents the Deaths by underlying cause according to the standard and original certificate and comparison of figures deriving from the two certificates.

9. Table 3. Last column (Change in cause – specific mortality fraction (%)): if the number of deaths for the original certificate is greater than for the standard, why the sign of the indicator is negative? According to the formula (line 114) I would expect a positive sign (and vice-versa for positive values).

10. Ln 171-177: This part presents the results for men and women (is it correct?), but it seems a repetition of the above comment. I suggest to mention that for most causes the results are comparable in the two genders and successively to focus only on chapters where there are differences.

11. Ln 177: Fig 1 is a table, it should be considered table 4.

12. Ln 182: The term validity in this context is not clear (the description of this table should be similar to table 2 which has similar content).

13. Figure 1 (should be table 4): the columns’ titles for women are different than for men and they have different order.

14. Ln 207-209: The authors report underestimation for ‘circulatory system disease’ but the results seem to show that the original death certificate overestimates them. The same occurs for ‘diseases of respiratory system’. This term should be used when the original death certificate, which is under evaluation, detects more cases than the standard. The use of the term underestimation considering the false negatives is misleading. I would suggest to use the CSMF, which is calculated but little used in comments.

15. Ln 210: The result about ‘pregnancy, childbirth and puerperium’ is very counterintuitive, since death certificates are often considered to underestimate maternal mortality, while examining additional documentation should allow to detect more accurately the cases. What is the authors’ opinion on this result? In addition, it is surprising that there are 7 cases classified in chapter XVII (Symptoms and signs) by the standard certificates which is based on additional documentation. Is there some explanation for these cases?

16. Ln 216-218: Could authors rephrase this sentence? It is not very clear.

17. Ln 224: Are there differences for deaths occurred in hospital compared to other places? Are there differences for source of determination of the cause of death? If possible, some comments could be provided in the discussion (even if not provided in the results).

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes:  Francesco Grippo

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: renamed_26069.docx

Attachment

Submitted filename: ValidityCauseDeathAuthors.docx

PONE-D-23-33548R1Accuracy of Information on the Underlying Cause of Death: An Analysis in Colombia during the COVID-19 Pandemic in 2021PLOS ONE

Dear Dr. Berrio-Parra,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Hereby I provide the main concerns:

*Originality and Context: Please provide a clearer rationale for the study's significance within the existing literature.

*Methodological Clarity and Ethical Considerations: There are several methodological and ethical considerations that need to be addressed. The temporal scope of mortality records needs clarification, along with details on informed consent procedures, mitigation of biases during data review, and anonymization processes. Please ensure transparency and provide clear explanations for these aspects in the Methods section.

*Engagement with Data Sources: Clarify the decision to use data from the National Institute of Health (INS) instead of the National Administrative Department of Statistics (DANE).

*Statistical Analysis and Results: Provide a more detailed explanation of the statistical analysis, including the calculation of false positives and negatives and the rationale behind using kappa statistics solely for establishing validity. Correct any errors in the presentation of results, such as those regarding circulatory diseases.

*Representativeness and Underestimation: Substantiate claims about underestimation of causes of death, especially for rare events, and clarify the representativeness within the sample.

*Grammar and style review: Please perform a thorough revision of the manuscript to meet PLOS ONE Criteria for Publication.

Please submit your revised manuscript by May 24 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Andres Mauricio Acevedo-Melo, M.D.

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #4: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #4: Partly

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: (No Response)

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: (No Response)

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: (No Response)

Reviewer #4: No

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #4: The paper is interesting but a paragraph in the Methods section is needed providing the context of the study, specifically addressing the following points of death certification in Colombia:

1) Who fills the death certificates (this is already reported in Discussion, but a brief anticipation is needed in Methods)

2) Which type of death certificate is adopted: exactly the International Death Certificate from the WHO, or an adaption?

3) In national mortality statistics, how the underlying cause of death is selected: by trained coders, or by a software as in the US (ACME) or in Europe (IRIS)?

4) Which version of the ICD-10 was adopted in the study period?

Moreover, in Methods authors should clearly specify the following:

1) What are they comparing, e.g. the disease on the underlying cause line in the death certificate compiled by the certifier, or the underlying cause of death selected according international coding rules by a coder or by software? This might not be important for the gold standard, since in a “perfect” death certificate the two conditions should coincide, but is relevant for the original death certificate: as an example, in a death certificate reporting pulmonary embolism in part I and cancer in part II, the latter (and not pulmonary embolism) will be selected according to international coding rules as the CoD

2) Authors should explicitly state already in Methods that they are measuring agreement at the chapter level of the ICD-10

Results need some improvement: a footnotes in all Tables with an abbreviated description of chapter numbers (e.g. “Infectious” for chapter I). Moreover, in page 11, last lines, an error was left from the previous version (circulatory diseases exhibited higher percentages of false positives, not of false negatives)

Lastly, authors should carefully check for typos through the manuscript: only as a first example, at the end of Methods in the Abstract, “…. and calculate kappa value were utilized” should be replaced with “… and kappa value were utilized …..”

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: No

Reviewer #4: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: PONE-D-23-33548_R1_reviewer.pdf

Attachment

Submitted filename: PONE-D-23-33548R1-Review.pdf

PONE-D-23-33548R2Accuracy of Information on the Underlying Cause of Death: An Analysis in Colombia during the COVID-19 Pandemic in 2021PLOS ONE

Dear Dr. Berrio-Parra,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Sep 19 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Andres Mauricio Acevedo-Melo, M.D.

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

Dear Authors,

Thank you for submitting a revised version of your manuscript. Before it can be considered for publication, there are a few issues that need to be addressed. Please review the following main points, which are related to PLOS ONE publication criteria and highlight the concerns raised by the reviewers:

*Conclusions Supported by Data: Your manuscript uses the terms “accuracy” and “agreement” interchangeably. However, it is important to note that kappa statistic measures agreement, while false positive and negative rates estimate discrimination. Additionally, the distinction between “original” and “standard” death certificates, role of participating institutions and custodianship need clarification. Please revise these concepts throughout the manuscript to ensure clarity. Also, provide clear descriptions and interpretations of measures such as the Change in Cause-Specific Mortality Fraction (CCSMF) and the Bland-Altman diagram axis and legends.

*Clarity and Language: The manuscript should be written in clear, standard English. Please conduct a thorough English language review and address the mistakes highlighted by the reviewers. The following typos and issues were also noted: the word “Charts” is used in the abstract submission, while “records” is used in the manuscript abstract; a repeated sentence appears on lines 35-38; “offres” should be corrected to “offers” on line 56; and the acronym “IRIS” on line 112 should be explained. Please also review and correct subject-verb agreement and coherence issues, particularly between lines 280 and 284.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The manuscript needs clarity on data sources, terminology, and the impact of the COVID-19 pandemic on death certificate accuracy.

Reviewer #4: Author improved the manuscript. Some minor revision is still needed

1) Abstract: please remove chapter numbers ("circulatory system diseases" is enough for the Abstract, without adding "Chapter IX")

2) Careful editing is needed, only as an example Introduction, page 3, row 56: please change "offres" with "offers"

3) Please correct Table 1: under "Reviewer underlying cause", change the right column from "Diagnosis X" to "The others (or better, "All other diagnoses"), and check all the following formulas

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: No

Reviewer #4: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: PONE-D-23-33548R3 Review Jul 28 2024.pdf

PONE-D-23-33548R3Agreement of information on the underlying cause of death: an analysis in Colombia during the COVID-19 pandemic in 2021PLOS ONE

Dear Dr. Berrio-Parra,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Dec 20 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Pasyodun Koralage Buddhika Mahesh

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #4: All comments have been addressed

Reviewer #5: (No Response)

Reviewer #6: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #4: Yes

Reviewer #5: Yes

Reviewer #6: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #4: Yes

Reviewer #5: Yes

Reviewer #6: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #4: Yes

Reviewer #5: Yes

Reviewer #6: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #4: Yes

Reviewer #5: Yes

Reviewer #6: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #4: Authors addressed my previous comments

Reviewer #5: The document is a manuscript draft discussing the agreement of information on the underlying cause of death in Colombia during the COVID-19 pandemic in 2021.

The key findings of the study by the authors are:

The agreement between the original and gold standard death certificates, categorized according to ICD-10 chapters, was 74%. Higher levels of agreement were observed for "codes for special COVID-19 situations" (kappa=0.84) and neoplasms (kappa=0.84). COVID-19 was considered the underlying cause of death in 29.8% of the original certificates and 31.8% of the gold standard certificates, with a very good agreement (kappa=0.84). The proportion of agreement for most causes of death remained consistent between genders, with 73% agreement in men and 75% in women.

The study recommends robust educational and training initiatives, the development of comprehensive guidelines, ongoing monitoring, the use of automated systems, and formalised review processes to improve the death certification process in Colombia.

I congratulate the authors for selecting this all-important topic and conducting the study to identify potential misclassifications in the cause of death data. Using Iris automated coding software for mortality coding has ensured the correct and consistent application of coding rules to select the underlying cause of death.

My primary concern is selecting the broad categories (ICD Chapter level) to compare the original UCOD with the gold standard. Grouping causes of death at the ICD chapter level can be considered a broad categorisation, which may limit the granularity and specificity of the analysis. Here are some points to consider:

1. ICD chapters encompass a wide range of diseases and conditions. For example, the chapter on "Diseases of the circulatory system" includes various conditions such as myocardial infarction, strokes, and hypertension. Grouping at this level may mask discrepancies in specific causes of death within the chapter.

2. More detailed categorization, such as using ICD-10 codes at the sub-chapter or individual code level, could provide a clearer picture of the accuracy and quality of cause of death data. It would allow for identifying specific areas where misclassification or inaccuracies are more prevalent. The authors could easily use the mortality tabulation lists (General mortality 1 -103 causes or General mortality 2 – 80 causes) provided in ICD-10 for disease categorisation.

3. Detailed cause of death data is crucial for designing targeted public health interventions. Broad categories may not provide the necessary details to inform specific health policies and resource allocation.

4. The study acknowledges the limitations of using broad categories and suggests that a more detailed analysis could provide additional insights. However, the broad categorisation might have been chosen for practical reasons, such as data availability or analysis feasibility.

5. Future studies could benefit from a more detailed examination of cause of death data using specific ICD-10 codes. This would enhance the understanding of the quality of death certification and help identify specific areas for improvement.

In summary, while the broad categorization at the ICD chapter level provides a general overview of the agreement between original and gold standard death certificates, a more detailed analysis would offer greater specificity and potentially more actionable insights.

Another issue is that he chapter-level broad categories of causes of death could include garbage codes (unusable codes for public health decision-making) of medium impact. These are known as ‘Level 3 (medium)’ garbage codes, meaning codes with important implications: causes for which the true underlying cause of death is likely to be within the same ICD chapter. For instance, ‘unspecified cancer’ still provides enough information to know if the UCOD was due to cancer. However, knowledge about the site of cancer is important for public health policy since different strategies are applied for different types (sites) of cancer (i.e. breast versus lung cancer).

The categorization of garbage codes into levels, particularly Level 3 garbage codes, is a useful approach to understanding the quality of cause of death data. Level 3 garbage codes refer to causes of death where the actual underlying cause is likely within the same ICD chapter.

In summary, since the study's use of broad categories aligns with the concept of Level 3 garbage codes, it still needs to be revised in terms of specificity and potential misclassification. Recognizing and addressing the presence of Level 3 garbage codes can help improve the quality of cause-of-death data and enhance the utility of mortality statistics for public health.

While the study methodology is generally sound, there are a few potential issues and limitations that could be addressed:

1. As previously mentioned, grouping causes of death at the ICD chapter level is quite broad and may mask specific discrepancies within those chapters. More granular categorization could provide a clearer picture of the accuracy and quality of cause of death data.

2. Determining the gold standard cause of death relied heavily on medical records and, when necessary, interviews with relatives or witnesses. Medical records may not always be complete or accurate, and interviews can introduce recall bias.

3. Although the health professionals were trained and supervised, there is still a risk of observer bias in interpreting medical records and assigning causes of death.

4. The study used data from the DANE 2021 mortality database and medical records. It did not include other potential sources of information, such as autopsy reports, which could provide additional accuracy in determining the cause of death.

5. The study focused on 92 municipalities, which may only partially represent part of the country. Geographical and demographic variability in the quality of death certification could exist, but this is not captured in this sample.

6. The study was conducted during the COVID-19 pandemic, which could have affected the accuracy of death certification due to the high volume of deaths and the strain on healthcare systems. This context should be considered when interpreting the results.

7. While the study received ethical approval and took measures to anonymise data, obtaining consent from relatives or witnesses for interviews could introduce ethical concerns, especially in sensitive or traumatic deaths.

8. The findings may not be generalizable to other countries or regions with different healthcare systems, death certification processes, and levels of training for health professionals.

9. Although the study mentions that health professionals were trained and supervised, it needs to provide detailed information on the extent and quality of this training. Inadequate training could affect the reliability of the gold standard in determining the cause of death.

10. While the sample size is substantial, the study does not discuss potential selection bias in the choice of municipalities and deaths included in the sample. Selection bias could affect the representativeness of the findings.

Addressing these issues in future research could enhance the robustness and reliability of the findings and provide more detailed insights into the quality of cause of death data.

Reviewer #6: This article aims to estimate the agreement of the underlying Cause of Death in the original death certificate, compared with a gold standard certificate based on information from medical records and relatives, in Colombia during 2021. The authors have used summary measures of agreement such as the proportion of agreement, Kappa, CCSMF and graphical methods such as the Bland-Altman plot. While commending their efforts, the following comments are given with the hope that these will benefit the authors.

1. Please elaborate on the two-stage sampling method a bit more. As an example, it is mentioned that in the second stage, 92 municipalities were selected. It would be better to add that these 92 were selected out of how many. Also, what method was used in this random sampling (i.e. whether done by computer-generated random numbers etc.)? Furthermore, it is mentioned that 33 territorial entities were “considered” in the first stage. It would be nicer to elaborate a bit more on how this consideration was done.

2. Please give a reference to the sample size calculation formula/ software that was used.

3. In the sample size calculation, please define what variable was taken as “expected proportion of 70%” (i.e. whether it is the expected proportion of agreement?). Was this figure selected based on some previous literature or by some other method?

4. Please add any rationale for choosing the design effect of 2.5

5. It is mentioned that “In cases where medical 143 records lacked sufficient information, interviews with relatives, acquaintances, or witnesses 144 were conducted". Please add a bit more details on who conducted these interviews, in what settings and what kind of information were asked from the respondent etc.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #4: No

Reviewer #5: No

Reviewer #6: Yes:  I.O.K.K.Nanayakkara

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

Agreement of information on the underlying cause of death: an analysis in Colombia during the COVID-19 pandemic in 2021

PONE-D-23-33548R4

Dear Dr. Berrio-Parra,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager®  and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Pasyodun Koralage Buddhika Mahesh

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #5: All comments have been addressed

Reviewer #6: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #5: Yes

Reviewer #6: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #5: Yes

Reviewer #6: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #5: Yes

Reviewer #6: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #5: Yes

Reviewer #6: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #5: Thank you very much for your responses. It is great to see that you have responded taken appropriate action to all my comments. They have provided detailed responses and made necessary adjustments to the manuscript to address the concerns raised. Here are the key points of how you addressed the comments:

1. ICD Chapters and Detailed Categorization

The authors acknowledged the broad categorisation issue and incorporated the ICD-10 mortality tabulation list 2 (80 causes) for a more detailed analysis. They included the results in the supplement (Table 1S) and discussed the limitations of broad categories in the manuscript.

2. Granular Categorization

They agreed with the need for more detailed categorization and implemented it in the revised version. In the discussion section, they also discussed the impact of using broad categories and the benefits of detailed analysis.

3. Public Health Interventions

The authors concurred with the importance of detailed cause-of-death data for public health interventions and acknowledged the limitations of broad categories. They emphasized the need for future studies to use more detailed categorisation.

4. Garbage Codes

The authors recognized the importance of garbage codes and mentioned that Level 3 garbage codes were found in 2.84% of the basic CoD certifications, with most being "unspecified cancer."

5. Gold Standard and Observer Bias

They addressed the potential limitations of using medical records and interviews for the gold standard and acknowledged the risk of observer bias. They included these points in the discussion section as limitations.

6. Autopsy Reports

The authors clarified that autopsies are rarely performed for natural causes in Colombia, which is why they were not included in the study.

7. Geographical and Demographic Variability

They explained the probability sampling method used to ensure representativeness and provided additional details on the sampling process.

8. COVID-19 Pandemic Context

The authors included the context of the COVID-19 pandemic in the discussion, acknowledging its potential impact on the accuracy of death certification.

9. Ethical Concerns

They noted that no ethical concerns arose during the data collection process and emphasised the measures taken to ensure ethical standards.

10. Training of Health Professionals

The authors provided more details on the training given to health professionals by a DANE vital statistics expert.

11. Selection Bias

They explained the random sampling method used to minimise selection bias and provided details on the sampling process.

Overall, you have made comprehensive revisions to address my comments, enhancing the robustness and reliability of their findings. Therefore, I recommend that this manuscript be suitable for publication in your journal. Congratulations!

Reviewer #6: Thank you for submitting the revised manuscript and for addressing the reviewer’s comments thoughtfully. After carefully reviewing the changes, it is clear that the authors have made significant improvements, especially in clarifying the methodology and refining the discussion of the results.

The study is important, and the results contribute valuable insights to the understanding of cause of death classification in Colombia.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #5: Yes:  U S H GAMAGE

Reviewer #6: Yes:  I.O.K.K.Nanayakkara

**********