https://orcid.org/0000-0002-7119-8187
Figures
Abstract
Objective
The aim of this study was to emulate the MERINO trial of piperacillin-tazobactam vs meropenem for the definitive treatment of bloodstream infection (BSI) caused by ceftriaxone-nonsusceptible E coli or Klebsiella spp.
Methods
Data from an observational study of BSI and a randomised controlled trial of a rapid diagnostic in BSI were used to emulate the MERINO trial. The primary outcome of the emulated trial was 28-day mortality after blood culture. Outcomes were compared using logistic regression adjusted for propensity score for emulated intervention.
Results
Of the 6,371 observational study and RCT participants, 1,968 had a bloodstream infection with E. coli or Klebsiella spp. of which 121 met the eligibility criteria. In the emulated trial, a total of 14/82 patients (17.1%) allocated to piperacillin-tazobactam met the primary outcome compared with 6/39 (15.4%) in the meropenem group (unadjusted odds ratio 1.13 (95% CI 0.40 to 3.21)). After adjustment for propensity score, the odds ratio increased to 1.31 (95% CI 0.40 to 4.26). This difference is in the same direction but of a smaller magnitudethan observed in the MERINO trial, where 30-day mortality was met by 23/187 patients (12.3%) in the piperacillin-tazobactam and 7/191 (3.7%) in the meropenem group (unadjusted odds ratio of 3.69 (95% CI 1.48 to 10.41)).
Conclusions
The mortality rate in an emulated trial population was more than double the mortality rate in the MERINO trial. The methodology used attempts to address the concern that previous results could be explained by biases such as selection bias and uncontrolled confounding and provides information on how a trial such as the MERINO trial may have performed in the NHS.
Citation: Evans RN, Harris J, Rogers CA, MacGowan A (2022) Emulating the MERINO randomised control trial using data from an observational cohort and trial of rapid diagnostic (BSI-FOO). PLoS ONE 17(5): e0268807. https://doi.org/10.1371/journal.pone.0268807
Editor: D William Cameron, University of Ottawa, CANADA
Received: November 6, 2020; Accepted: May 8, 2022; Published: May 20, 2022
Copyright: © 2022 Evans et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are available at the University of Bristol data repository, data.bris, at https://doi.org/10.5523/bris.2jpy7mh519ono2exurvl84zwzl.
Funding: This National Institute for Health Research (NIHR) Programme Grants for Applied Research funded the BSI-FOO and RAPIDO studies (RP-PG-0707-10043). The British Heart Foundation and NIHR Bristol Biomedical Research Unit for Cardiovascular Disease and North Bristol Charitable Funds funded some staff time (JH, CR, RE). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Extended-spectrum β-lactamase (ESBL) producing bacteria are a frequent cause of bloodstream infection (BSI). Carbapenems have been regarded as the antibiotic of choice for treatment of infections caused by ESBL producers [1]. However, it has been shown that increased use of carbapenems is associated with increased incidence of carbapenem resistant Enterobacteriaceae [2]. Alternative treatments are needed to help contain the spread and frequency of carbapenem resistance. β-Lactam/β-lactamase inhibitor (BLBLI) combination antibiotics, such as piperacillin-tazobactam, have been considered a carbapenem-sparing option for treatment of ESBL producers [3, 4]. There have been a number of observational studies that have shown that BLBLIs are an effective treatment for infections caused by ESBL producers [5–9], and recent reviews have shown this to differ depending on the infection severity [10, 11]. However, observational analyses are subject to bias and results from a recent randomised controlled trial (RCT), the MERINO trial [12], were not consistent with results published in these observational studies. The MERINO trial is a recent international RCT to determine whether definitive therapy with piperacillin-tazobactam is noninferior to meropenem in patients with BSI caused by ceftriaxone-non-susceptible E.coli or K.pneumoniae [12]. A noninferiority margin of 5% was used. The MERINO trial concluded that definitive treatment with piperacillin-tazobactam did not result in noninferior 30-day mortality compared to definitive treatment with meropenem.
An RCT is considered the gold standard design in clinical research, however, they are not always financially or ethically feasible to conduct. Therefore, an observational approach is often used but these studies are subject to bias and confounding. Emulating a target trial is an approach designed to “mimic” trial practice using observational data and if successful, should yield similar results [13].
The aim of this study was to use data from an observational study and a trial of a rapid diagnostic to emulate the MERINO trial to explore whether an emulated trial would yield consistent results using data from the UK.
Methods
Study design
Patient-level data from the two studies performed as part of the BSI-FOO Programme [14, 15] were used to emulate the MERINO trial eligibility criteria, treatment strategy, and statistical analysis. The two studies were part of the same NIHR research programme and data collection was similar. Therefore, to maximise the potential sample size of the emulated trial, data from both studies were used. One was the BSI-FOO observational study and the other the RAPIDO RCT (trial registration ISRCTN97107018). The BSI-FOO observational study was a multicentre cohort study of 1,903 hospitalised patients with a BSI across seven NHS acute hospital trusts in England and Wales conducted between November 2010 and May 2012 with the primary aim of identifying modifiable risk factors for 28-day mortality. Adults (≥18 years old) receiving in-patient NHS hospital care and having a clinically significant BSI caused by six key pathogens: 1) methicillin-resistant Staphyloccos aureus (MRSA); 2) methicillin-susceptible S. aureus (MSSA); 3) non-Extended-spectrum beta-lactamase (ESBL)-producing Escherichi coli; 4) any ESBL-producing member of the family Enterobacteriales; 5) Pseudomonas aeruginosa; 6) any species of Candida, were included. RAPIDO was a multicentre open parallel group (1:1) RCT comparing two approaches to the identification of the causative microorganism(s) of BSI in hospitalised adult patients. The RAPIDO trial took place in seven NHS acute hospital trusts in England and Wales between July 2012 and August 2014. Date and time 0 was the date and time of first positive blood sample confirming BSI in both the observational study and the RCT. Full details of the inclusion criteria and study designs are given in the study results publications [14, 15].
The research programme on which this work is based was approved by Southwest Research Ethics Committee (10/HO102/51). The National Information Governance Board approved the use of routinely-collected patient data without specific consent for the BSI-FOO observational study and collection of full data for patients who died before being approached for consent in RAPIDO. North Bristol NHS Trust acted as Sponsor.
Study population
The MERINO trial inclusion/exclusion criteria were applied to BSI-FOO and RAPIDO participants (Table 1). Patients were included if they had a BSI with E. coli or Klebsiella spp. that was resistant to Ceftriaxone and/or Cefotaxime and susceptible to both meropenem and piperacillin-tazobactam and treatment was started within 72 hours of blood culture (date and time 0 for BSI-FOO and RAPIDO). Patients that would have otherwise been eligible but did not start any of the study drugs (meropenem or piperacillin-tazobactam) within the 72 hours window were considered ineligible and excluded from the emulated trial population. All participants were aged 18 years and over. Informed consent was not required for the BSI-FOO observational study and RAPIDO patients who died before being approached for consent, so it was not possible to replicate consent for this population, but all surviving RAPIDO participants provided written informed consent to join the trial. Exclusion criteria are given in Table 1.
Intervention
The trial interventions in the MERINO trial were treatment with piperacillin-tazobactam or meropenem. These were to be administered for a minimum of 4 days and maximum of 14 days, with duration determined by the treating clinician. In the emulated trial population patients were assigned to a “emulated intervention” based on their treatment timeline and allocated to the first study drug received. The start of follow-up was defined as the date in which the patient started their first dose of their assigned intervention.
Outcome measures
The primary outcome of the MERINO trial was all-cause mortality at 30 days after randomisation. It was not possible to analyse 30-day mortality for the emulated trial as follow-up in BSI-FOO and RAPIDO was limited to 28 days, where start of follow-up was defined as the date the blood sample was taken. All patients started their emulated intervention within 3 days of blood sample; therefore, 25-day mortality was analysed to ensure full follow-up was available for all patients.
Statistical analyses
The MERINO trial analysis population was defined as any randomised participant receiving at least 1 dose of the allocated drug. This was supported by an analysis of the per-protocol population. By definition, in the emulated trial population, all participants received at least one dose of allocated drug and were therefore included in the primary analysis population. We did not emulate the per-protocol analysis as few participants received allocated treatment for the required four days.
Continuous data were summarised using mean and standard deviation (or median and interquartile range (IQR) if distributions were skewed) and categorical data as numbers and percentages. Demographics, comorbidities and medical history were summarised by emulated intervention. Standardised mean differences were calculated to quantify imbalances in baseline characteristics by the treatment group [16]. Mortality over 25-days was summarised by emulated intervention using inverse probability weighted survival curves (weighted according to the inverse probability of treatment received, see below for further details of propensity score) to show adjusted survival graphically [17].
To emulate the trial analyses, absolute risk differences were calculated using generalised linear models. As the emulated trial was not randomised, potential confounding factors needed to be accounted for in the analysis. Propensity score models were developed using a logistic regression model with emulated intervention as the outcome. Variables included in the propensity score model were age and sex and any potential confounders based on clinician expertise. These were specified a priori. Factors included in the propensity score model were: centre, age, sex, temperature at time 0, neutrophil count on day 0 or closest, systolic blood pressure on day 0 or closest, on IV fluids at day, on ventilation at day 0, cerebrovascular disease, Charlson score and source of infection. The number of participants and deaths in each emulated intervention group was examined within strata defined by propensity score quantiles. Participants in strata for which there were no participants or deaths in either group were excluded to ensure the analyses were restricted to participants eligible to receive either treatment strategy, ensuring the assumption of positivity was met.
Convergence was not achieved when fitting an adjusted generalised linear model and therefore outcomes were compared using logistic regression adjusted for the propensity score. Unadjusted odds ratios were calculated for the MERINO trial to provide a comparison. Propensity scores were modelled using restricted cubic splines with 3 knots at 10th, 50th and 90th percentiles to capture potential non-linear associations with the outcome. Missing values were imputed with age- and sex-adjusted averages.
Three sensitivity analyses were carried out: (a) imputing missing categorical values with worst case values i.e. disease present; (b) propensity score model using restricted cubic splines at 25th, 50th and 75th percentiles to assess the robustness of the results to the location of knots; (c) excluding participants that switch to the other intervention during follow-up.
Model fit was assessed using standard methods. Meropenem was the reference group in all analyses. Results are reported as effect estimates with 95% confidence intervals (CI). In all tables missing data are described in footnotes.
All analyses were performed in Stata version 16.0 (StataCorp, LP, College Station, TX, US).
Results
Of the 6,371 BSI-FOO and RAPIDO participants, 1,968 had a BSI with E. coli or Klebsiella spp. of which 163 had proven non-susceptibility to cephalosporins and proven susceptibility to meropenem and piperacillin-tazobactam. Of these, 34 were not in receipt of meropenem or piperacillin-tazobactam within 72 hours of blood culture. Of the remaining 129 participants, 4 repeat episodes and 4 polymicrobial infections were excluded, thus 121 met the eligibility criteria (S1 Fig in S1 File) and were included in the analysis population. No observations were excluded based on propensity scores/confounders.
Intervention
Of the 121 participants who met the emulated trial eligibility criteria, 82 were assigned to piperacillin-tazobactam and 39 to meropenem, according to their first study drug received. The median time to receipt of study drug was longer in the meropenem group (38 hours (IQR 8, 54)) compared to the piperacillin-tazobactam group (6 hours (IQR 0, 19)) and duration of allocated treatment was also longer in the meropenem group (7 days (IQR 4, 8) vs. 3 days (IQR 2, 5)). Of those allocated to meropenem, 31/39 (79.5%) were in receipt of their study drug for four days (MERINO trial per-protocol) and of those allocated to piperacillin-tazobactam, 38/82 (46.3%) were in receipt of their study drug for four days. After the first dose of emulated intervention, 1/39 (2.6%) participants allocated to meropenem switched to piperacillin-tazobactam, conversely, 39/82 (47.6%) switched from piperacillin-tazobactam to meropenem (S1 Table in S1 File).
Demographics
Demographic characteristics and medical history are shown by emulated intervention in Table 2 and for the MERINO trial vs. emulated trial in S2 Table in S1 File and inverse probability weighted in S3 Table in S1 File. Overall, the participant characteristics were similar to the MERINO population with the exception of Charlson comorbidity index which was slightly higher (3.0 vs. 2.0) and moderate-severe renal dysfunction which was present in a higher proportion of participants (61% vs. 16%) in the emulated trial population.
Primary outcome
The overall 25-day mortality rate was 20/121 (16.5%) compared to 30/378 (7.9%) 30-day mortality in the MERINO trial population [12]. Inverse probability weighted Kaplan-Meier curves displaying time to death according to emulated intervention group are shown in Fig 1. In this emulated trial, a total of 14/82 participants (17.1%) allocated to piperacillin-tazobactam met the primary outcome of all-cause mortality at 25 days compared with 6/39 (15.4%) in the meropenem group (risk difference 1.7%, 95% CI -12.26 to 15.64). The corresponding unadjusted odds ratio is 1.13 (95% CI 0.40 to 3.21). After adjustment for propensity score, the odds ratio increased to 1.31 (95% CI 0.40 to 4.26). Sensitivity analysis gave similar results (Table 3). These differences are lower than observed in the MERINO trial, where 30-day mortality was met by 23/187 participants (12.3%) in the piperacillin-tazobactam and 7/191 (3.7%) in the meropenem group with an unadjusted risk difference 8.6 (95% CI 3.0 to 14.5) and odds ratio of 3.69 (95% CI 1.48 to 10.41). Sensitivity analyses gave similar results.
Discussion
The overall mortality rate in the emulated trial was more than double the mortality rate in the MERINO trial but similar to mortality rates reported in other observational studies [5, 6]. Although the effect size was smaller in the emulated trial (OR = 1.31 compared to 3.7 in the MERINO trial), the 95% confidence interval for the emulated trial estimate (95% CI 0.4 to 4.26) includes the estimated odds ratio from the MERINO trial which suggests potential consistency given the smaller sample size in this study.
There are several differences in the study design and population characteristics that could explain the lower mortality rates observed in the MERINO trial compared to the emulated trial. Firstly, there are differences in the demographics, comorbidities and severity of illness between the emulated trial population and the MERINO trial population. Secondly, the process of obtaining patient consent in MERINO could result in the sickest patients not being captured. Consent was not required for the BSI-FOO observational study and therefore all eligible patients were included. In addition, the exclusion of individuals who were deemed unlikely to survive beyond 96 hours in MERINO may have also resulted in the sicker patients who would otherwise be eligible for the trial being excluded which could lead to an underestimation of the true mortality rate. This was acknowledged by the authors as a limitation of the study. We did not impose the 96-hour restriction in the trial emulation as this could have introduced survival bias. Finally, over half of the infections in the MERINO trial were urinary tract which are known to be more responsive to treatment. The treatment effect may differ across different levels of infection severity/presence of comorbidities e.g. piperacillin-tazobactam may be inferior to carbapenems in patients with severe infections but non-inferior in less severe infections such as urinary tract infections [10, 11], however we did not have a large enough sample size to explore this. Further research is required to investigate this, but this may in part explain the conflicting results published in the MERINO trial to other observational studies where the populations and severity of illness are likely to differ.
Strengths and limitations
One of the strengths of this study is the trial emulation approach used in the design and analysis. Trial emulation using observational data ensures that eligibility criteria and assumptions are explicit before analysis and minimises common biases that arise in observational data analyses. In addition, a number of the published observational studies exclude patients who are not in receipt of either intervention for >48hours [7–9], meaning patients who die within 48 hours of receipt treatment are excluded from the population which gives potential for introducing survival bias. Applying trial emulation methods enabled us to include all patients who would be eligible for a trial, without using data after start of follow-up in the inclusion/exclusion criteria, therefore minimising survival bias. In addition, no UK patients were recruited to the MERINO trial, so applying trial emulation methods to UK data provides information on how a trial such as the MERINO trial may have performed in the NHS and the addition of UK data to the literature adds to the generalisability of the currently published work.
There are several limitations to this study. First, it was not possible to emulate all elements of the MERINO trial. The MERINO trial had 30-day follow-up, we did not have 30-day follow-up for all BSI-FOO and RAPIDO participants and therefore we analysed 25-day mortality. However, this is unlikely to have a significant impact as later deaths are less unlikely to be a result of treatment. Upon examination of the Kaplan-Meier curves, there is a larger difference in mortality rate up to 14 days. We did not formally compare mortality rates at 14 days due to the small number of events and lack of statistical power, however we feel that 14-day mortality may be a more clinically meaningful outcome for studies investigating mortality in BSI as it is the time period most reflective of death attributable to suboptimal therapy. In addition, we did not explore other outcomes other than mortality as this was beyond the scope of the trial emulation, however other outcomes such as composite outcome of treatment failure and mortality may merit analysis in future research.
Secondly, it was not possible to emulate the per-protocol analysis fully as few participants received allocated treatment for four days, and restricting analyses to those who are in receipt for four or more days would introduce immortal time bias (bias induced by a period of follow-up during which, by design, the outcome cannot occur). Our approach made it hard to attribute differences in the intention-to-treat analysis because some participants received both treatments with many of the participants assigned to piperacillin-tazobactam swapping to meropenem leading to contamination of drug exposure. This compromises our ability to draw any firm conclusions from this study. We excluded participants who switched to the other intervention during follow-up in a sensitivity analysis and this gave similar results, although it is worth noting that this sensitivity analysis had a smaller sample size, and this analysis is likely to be subject to selection bias as switching is usually related to prognosis. In addition, participants’ empirical treatment and treatment pathways other than the “allocated” intervention were not controlled for so any observed differences could be attributed to the effects of empirical therapy or treatments received after classification of “trial drug”. There were also differences in the time to receipt of “allocated” treatment between the two groups.
Finally, it was not possible to emulate blinding, so the validity of our estimates depends on the assumption that all confounding factors were correctly adjusted for. We allowed for differences in baseline characteristics by adjusting for propensity score. However, due to the retrospective nature of the study we were only able to control for variables that had been collected and there is the risk that unmeasured confounders may impact the results.
In summary, the mortality rate in an emulated trial population was more than double the mortality rate in the MERINO trial and the difference between mortality rates in piperacillin-tazobactam and meropenem was weaker but the direction of effect was consistent with the MERINO trial. Our findings suggest that the discrepancies between the MERINO trial and observational studies estimates could be partly explained by differences in the populations and also due to the bias that arises from observational studies e.g. survival bias and bias from unmeasured or uncontrolled confounding. This methodology attempts to address the concern that previous results could be explained by such biases and provides information on how a trial such as the MERINO trial may have performed in the NHS.
Supporting information
S1 File. Emulating the MERINO trial—Supplementary material.
https://doi.org/10.1371/journal.pone.0268807.s001
(PDF)
References
- 1. Vardakas KZ, Tansarli GS, Rafailidis PI, Falagas ME. Carbapenems versus alternative antibiotics for the treatment of bacteraemia due to Enterobacteriaceae producing extended-spectrum beta-lactamases: a systematic review and meta-analysis. The Journal of antimicrobial chemotherapy. 2012;67(12):2793–803. pmid:22915465
- 2. McLaughlin M, Advincula MR, Malczynski M, Qi C, Bolon M, Scheetz MH. Correlations of antibiotic use and carbapenem resistance in enterobacteriaceae. Antimicrobial agents and chemotherapy. 2013;57(10):5131–3. pmid:23836188
- 3. Harris PN, Tambyah PA, Paterson DL. beta-lactam and beta-lactamase inhibitor combinations in the treatment of extended-spectrum beta-lactamase producing Enterobacteriaceae: time for a reappraisal in the era of few antibiotic options? The Lancet Infectious diseases. 2015;15(4):475–85. pmid:25716293
- 4. Tamma PD, Rodriguez-Bano J. The Use of Noncarbapenem beta-Lactams for the Treatment of Extended-Spectrum beta-Lactamase Infections. Clinical infectious diseases: an official publication of the Infectious Diseases Society of America. 2017;64(7):972–80.
- 5. Gudiol C, Royo-Cebrecos C, Abdala E, Akova M, Alvarez R, Maestro-de la Calle G, et al. Efficacy of beta-Lactam/beta-Lactamase Inhibitor Combinations for the Treatment of Bloodstream Infection Due to Extended-Spectrum-beta-Lactamase-Producing Enterobacteriaceae in Hematological Patients with Neutropenia. Antimicrobial agents and chemotherapy. 2017;61(8). pmid:28584145
- 6. Gutierrez-Gutierrez B, Perez-Galera S, Salamanca E, de Cueto M, Calbo E, Almirante B, et al. A Multinational, Preregistered Cohort Study of beta-Lactam/beta-Lactamase Inhibitor Combinations for Treatment of Bloodstream Infections Due to Extended-Spectrum-beta-Lactamase-Producing Enterobacteriaceae. Antimicrobial agents and chemotherapy. 2016;60(7):4159–69. pmid:27139473
- 7. Ko JH, Lee NR, Joo EJ, Moon SY, Choi JK, Park DA, et al. Appropriate non-carbapenems are not inferior to carbapenems as initial empirical therapy for bacteremia caused by extended-spectrum beta-lactamase-producing Enterobacteriaceae: a propensity score weighted multicenter cohort study. European journal of clinical microbiology & infectious diseases: official publication of the European Society of Clinical Microbiology. 2018;37(2):305–11.
- 8. Ng TM, Khong WX, Harris PN, De PP, Chow A, Tambyah PA, et al. Empiric Piperacillin-Tazobactam versus Carbapenems in the Treatment of Bacteraemia Due to Extended-Spectrum Beta-Lactamase-Producing Enterobacteriaceae. PloS one. 2016;11(4):e0153696. pmid:27104951
- 9. Rodriguez-Bano J, Navarro MD, Retamar P, Picon E, Pascual A. beta-Lactam/beta-lactam inhibitor combinations for the treatment of bacteremia due to extended-spectrum beta-lactamase-producing Escherichia coli: a post hoc analysis of prospective cohorts. Clinical infectious diseases: an official publication of the Infectious Diseases Society of America. 2012;54(2):167–74. pmid:22057701
- 10. Aslan AT, Akova M. Extended spectrum β-lactamase producing enterobacteriaceae: carbapenem sparing options. Expert review of anti-infective therapy. 2019;17(12):969–81. pmid:31722185
- 11. Gutiérrez-Gutiérrez B, Rodríguez-Baño J. Current options for the treatment of infections due to extended-spectrum beta-lactamase-producing Enterobacteriaceae in different groups of patients. Clinical microbiology and infection: the official publication of the European Society of Clinical Microbiology and Infectious Diseases. 2019;25(8):932–42. pmid:30986558
- 12. Harris PNA, Tambyah PA, Lye DC, Mo Y, Lee TH, Yilmaz M, et al. Effect of Piperacillin-Tazobactam vs Meropenem on 30-Day Mortality for Patients With E coli or Klebsiella pneumoniae Bloodstream Infection and Ceftriaxone Resistance: A Randomized Clinical Trial. JAMA. 2018;320(10):984–94. pmid:30208454
- 13. Hernán MA, Robins JM. Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available. American journal of epidemiology. 2016;183(8):758–64. pmid:26994063
- 14. MacGowan A, Grier S, Stoddart M, Reynolds R, Rogers C, Pike K, et al. Impact of rapid microbial identification on clinical outcomes in bloodstream infection: the RAPIDO randomized trial. Clinical microbiology and infection: the official publication of the European Society of Clinical Microbiology and Infectious Diseases. 2020. pmid:32220636
- 15. Evans RN, Pike K, Rogers CA, Reynolds R, Stoddart M, Howe R, et al. Modifiable healthcare factors affecting 28-day survival in bloodstream infection: a prospective cohort study. BMC Infectious Diseases. 2020;20(1):545. pmid:32711452
- 16. Cohen J. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, New Jersey: Lawrence Erlbaum Associates; 1988.
- 17. Cole SR, Hernan MA. Adjusted survival curves with inverse probability weights. Computer methods and programs in biomedicine. 2004;75(1):45–9. pmid:15158046