PONE-D-20-40076

Can a local audit of antibiotic therapy in bacteremia provide useful feedback on how to improve clinicians’ empiric antibiotic choice? A retrospective cohort study

PLOS ONE

Dear Dr. Bai,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The reviewers have carefully evaluated the manuscript and commented in details. Please follow their comments and address them one by one. Make sure you terminology regarding therapy is consistent throughout the manuscript. Please provide details regarding coverage of empiric antibiotics that is insufficient and coverage that is excessively broad (superfluous). Make sure the title represents what is presented in the study. Make sure the limited external validity of the study is mentioned, as the reviewers recommended. 

Please submit your revised manuscript by 4-Mar-2021. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Dafna Yahav

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating in your ethics statement "Consent was not obtained as the study was a retrospective chart review and the data were analyzed anonymously." In your Methods section and the ethics statement in the online submission form, please clarify whether all data were also fully anonymized before you accessed them, and/or whether the IRB or ethics committee waived the requirement for informed consent.

3. Thank you for providing the date(s) when patient medical information was initially recorded. Please also include the date(s) on which your research team accessed the databases/records to obtain the retrospective data used in your study.

4. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors performed a cohort study of patients receiving antibiotic therapy for positive blood cultures and evaluate adequacy of coverage and risk factors for inadequate coverage. This is an important endeavor for antimicrobial stewardship programs as empiric appropriateness may often be overlooked because it is more feasible to review antibiotics later during the stay and/or evaluate overall antibiotic utilization data. The use of terminology to describe appropriateness/adequacy in this manuscript was somewhat confusing and could be standardized further. Since inappropriate antibiotic therapy usually includes both inadequate (coverage of empiric antibiotics is insufficient) and unnecessary (coverage is excessively broad), both should be addressed and appropriate terminology should be used throughout.

The following are some considerations for improvement:

ABSTRACT

1. "Correct" terminology implies that excessively broad empiric antibiotics are appropriate. Consider using terminology "adequate" and "inadequate" or similar.

Consider using a framework similar to that of NAPS for clear/standardized definitions: https://www.safetyandquality.gov.au/sites/default/files/2020-02/report_-_2018_hospital_naps.pdf (page 49)

2. Consider providing a difference between negative likelihood ratios for the clinical pathway vs. clinician decision with a confidence interval around the point estimate. Although numerically different, the confidence interval for both likelihood ratios appears quite wide and hard to appreciate a difference between the estimates.

INTRODUCTION

3. As above, please reconsider the use of the terms "correct" and "incorrect" coverage.

METHODS

4. Were only inpatients included, or could non-admitted patients also count (e.g., blood culture in ED)?

5. Were Staphylococcus lugdunensis cases excluded from analysis, as they would be part of CNST category?

6. How were polymicrobial blood cultures handled?

7. Since patient data were collected retrospectively, and patient status or information may change during the hospital stay, please indicate whether these data were from the empiric period (e.g., infectious source - suspected source could shift over time, severity of infection - this could also change over time) or from any point during the hospital stay.

8. A brief statement and/or reference may be helpful to justify why 24 hours was selected as the empiric period (e.g., as opposed to a longer duration). Does the hospital use any form of rapid bacteriological testing?

9. Correct empiric coverage lines 131-157 use various terms "appropriate" and "unnecessary". Suggest using consistent terminology throughout the manuscript. It was not clear until now that unnecessarily broad empiric therapy would be considered. It appears that the authors do not count this as incorrect, but rather a separate category that is not necessarily mutually exclusive. It is somewhat confusing that an antibiotic could be both "correct" but "unnecessary". Since unnecessary therapy (e.g., vancomycin when not needed) is different than inadequate therapy (e.g., no vancomycin when it is needed), these should be more clearly distinguished throughout.

10. Line 164 "team that administered antibiotics" is unclear. Does this mean the medical or surgical service that prescribed the antibiotic?

RESULTS

11. Consider adding 2 columns to Table 2 and/or 3 to show characteristics of "inadequate" and "inadequate" treatment events, as this may be helpful to visually highlight crude differences in the population (in addition to the multivariable model shown later).

12. Table 4 title indicates "inappropriate" empiric therapy. It is not clear if it refers to adequacy of coverage or unnecessary coverage or both?

13. Many antimicrobial stewards will be interested in the total proportion of inappropriate antibiotics ordered (accounting for both incorrect or inadequate coverage and unnecessary coverage). As such, the current % "correct" antibiotic coverage likely underestimates inappropriateness. Has the total inappropriate % been calculated?

DISCUSSION

13. Although useful, many hospitals particularly smaller facilities, will not have the resources or analytical expertise to perform such analyses and identify opportunities for improvement. This should be acknowledged or comments provided to consider how low resource settings would perform such an audit (e.g., is using crude data alone enough?).

Reviewer #2: The authors describe the results of an audit of antibiotic prescription practice and present a new diagnostic "tool" to improve antibiotic use (mainly to reduce un-necessary broad-spectrum antibiotic use). The main findings are that 72% of the empiric antibiotic courses were incorrect, and they have pointed out the variables associated with incorrect empiric treatment, which included prior antibiotic use, and unknown source of infection. The diagnostic accuracy tables are presented but their impact on antibiotic use is not described in this manuscript, neither the outcome of correct versus incorrect empiric treatment.

Comments:

1. The title is confusing - the study only demonstrates the results of a local audit and does not really answer the question presented in the title. Consider modifying it to reflect the study objectives and findings.

2. The univariable analysis includes only a small number of variables and only odds ratio are presented. I would be interested in a table that compares patients who were and were not treated correctly. This table should include variables that may be associated with correct empiric treatment, including many of the variables presented in Table 2 and 3. I would also recommend to add the antibiotic treatment that were used in such a table, as well as the pathogens treated and their resistance patterns.

3. Using diagnostic accuracy parameters (positive and negative likelihood ratios, PLR, NLR, respectively) is a nice idea, but when treating life threatening bloodstream infection (BSI), I believe that the PLR should be higher in any given diagnostic test than the one presented here, this should be addressed in the discussion.

In this case the results are only relevant to rule out each of the BSI evaluared. Taking Pseudomonas BSI as an example, 50% (6/12) of patients with pseudomonas BSI did not receive empiric antipseudomonal coverage. According to the results, using NLR 0.17 of healthcare or hospital infection means that if patients did not have hospital or healthcare associated infection then 39 patients would not receive anti-pseudomonal coverage and additional 1 patient would receive anti-pseudomonal coverage if the patients had hospital acquired infection.

In the discussion the authors claim that not having hospital acquired infection means that the patient will likely not need anti-pseudomonal coverage, but it really cannot be a stand-alone clinical decision tool. First, it is unkown from this study results the impact of using this diagnostic tool on clinical outcomes. Second, as the authors well noted, the results of this study cannot be used in other hospitals and external validation should be performed.

Finally, I think that the authors should address the limitations of using diagnostic accuracy tables for clinical decisions and provide more data about how combined risk factors modified NLR and PLR, even if only as a supplemental table.

Clinician's decision – are there any criteria that clinicians used? were there more than one clinician that evaluated these cases? it is very reasonable that clinicians used the exact same risk factors to reach decision from the same table, and these are very heterogeneous decisions. That's why I don't think it should be used to evaluate diagnostic accuracy.

Minor comments:

How were prior cultures defined? What was the time period before the BSI?

The terms "correct" or "appropriate" should be used uniformly and not interchangeably.

Can the authors describe what were the antibiotics that were provided inappropriately? for example, how many ESBL pathogens were covered by piperacillin-tazobactam and thus defined inappropriately treated (how many were susceptible to piperacillin-tazobactam)? how many SPICE pathogens were empirically covered by third generation cephalosporins and defined as inappropriate empiric therapy (how many were susceptible to third generation cephalosporins)?

Table 4 – remove ER

Page 14, line 216 – the fact that blood cultures were not drawn from the intravascular catheter does not preclude it from being the source of infection. I would not categorized it as "unknown source".

Table 2: please describe "other sources" in the footnotes.

Can the authors describe what were the cases where carbapenem was used un-necessarily?

What was the antibiogram of Pseudomonas bacteremia? how many of these BSI were susceptible to each antibiotics?

In the discussion would address the fact that guidelines were applicable only for ~40% of the cases.

I believe it would be helpful to the readers if the authors could demonstrate how the NLR can be applied by showing pre-test and post-test probabilities for each risk factor and associated PLR and NLR.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Local audit of empiric antibiotic therapy in bacteremia: A retrospective cohort study

PONE-D-20-40076R1

Dear Dr. Bai,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Dafna Yahav

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for making these modifications. I have no further suggestions/comments. Good luck with dissemination of this important work.

Reviewer #2: I thank the authors for their thorough revision and edits. All my comments were responded appropriately.

I have not other comments.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Bradley Langford

Reviewer #2: No