Beta-lactams are the mainstay for treating methicillin-susceptible Staphylococcus aureus (MSSA) infections complicated by bacteremia due to superior outcomes compared with vancomycin. With approximately 11% of inpatients reporting a penicillin (PCN) allergy, many patients receive suboptimal treatment for MSSA bacteremia.
Evaluate the cost-effectiveness of penicillin skin testing (PST) in adult patients with self-reported PCN allergy in an inpatient setting undergoing treatment for MSSA bacteremia.
A decision analytic model was developed comparing an acute care PST intervention to a scenario with no confirmatory allergy testing. The primary outcome was the incremental cost-effectiveness ratio (ICER) from the health-sector perspective over a 1-year time horizon using quality-adjusted life years (QALYs) as the measure for effectiveness. One-way and probabilistic sensitivity analyses were conducted to assess the uncertainty of the ICER estimation.
Over a 1-year time horizon, PST services applied to all MSSA bacteremia patients reporting a PCN-allergy would result in a cost per patient of $12,559 and 0.73 QALYs while no PST services would have a higher cost per patient of $13,219 and 0.66 QALYs per patient. This resulted in a cost-effectiveness estimate of -$9,429 per QALY gained. Varying the cost of implementing PST services determined a break-even point of $959.98 where any PST cost less than this amount would actually be cost saving.
Patients reporting a PCN allergy on admission may receive sub-optimal alternative therapies to beta-lactams, such as vancomycin, for MSSA bacteremia. This economic analysis demonstrates that inpatient PST services confirming PCN allergy are cost-effective for patients with MSSA bacteremia.
Citation: Mattingly TJ II, Meninger S, Heil EL (2019) Penicillin skin testing in methicillin-sensitive staphylococcus aureus bacteremia: A cost-effectiveness analysis. PLoS ONE 14(1): e0210271. https://doi.org/10.1371/journal.pone.0210271
Editor: Georg Häcker, Albert-Ludwigs-Universitat Freiburg, GERMANY
Received: July 3, 2018; Accepted: December 19, 2018; Published: January 7, 2019
Copyright: © 2019 Mattingly et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are available within the manuscript along with the Full Model and Input Data uploaded as a supporting information file.
Funding: This study was supported by an unrestricted educational grant by ALK, Inc. The funding source had no role in the design and conduct of the study; collection, management, and analysis of the data; or preparation, review, and approval of the manuscript.
Competing interests: TJM reports personal fees from Summit Therapeutics, G&W Laboratories, and Paratek Pharmaceuticals outside the submitted work. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Abbreviations: MSSA, methicillin-sensitive Staphylococcus aureus; PST, penicillin skin test; PCN, penicillin; PSA, probabilistic sensitivity analysis; ADE, adverse drug event; ABX, antibiotics; INP, inpatient; OUT, outpatient; QALY, quality-adjusted life year; ICER, incremental cost-effectiveness ratio
Staphylococcus aureus is a leading cause of bacteremia that is associated with high mortality rates and represents a significant burden to the healthcare system.[1, 2] Beta-lactams are the mainstay for treating methicillin-susceptible Staphylococcus aureus (MSSA) infections complicated by bacteremia due to superior outcomes compared with vancomycin.[3–7] However, approximately 11% of inpatients report a penicillin (PCN) allergy, limiting optimal treatment for MSSA bacteremia.
Penicillin skin testing (PST) assesses local reactions to the major and minor determinants of type I reactions with a negative predictive value of 97–99%.[9, 10] It is estimated only 1% of the general population is truly allergic to penicillin and that less than 10% of patients with penicillin allergy histories who received PST are found to be at risk for an acute allergic reaction. Therefore ruling out penicillin allergy through PST would allow for antibiotic optimization in the treatment of many infectious conditions including MSSA bacteremia. Unfortunately, availability of PST in an inpatient setting is limited in many facilities due to lack of time or personnel.[12, 13] The objective of this study was to estimate the cost-effectiveness of PST in adult patients in an inpatient setting undergoing treatment for MSSA bacteremia.
We developed a decision analytic model using Microsoft Excel (Santa Rosa, California) to evaluate the cost-effectiveness of an acute care penicillin skin testing (PST) intervention for all patients admitted with MSSA bacteremia who self-report an allergy to PCN compared with the standard of care scenario with no confirmatory allergy testing. Once a treatment decision was made, path possibilities included treatment success with no ADE, treatment success with ADE, treatment failure with no ADE, and treatment failure with ADE. Patients receiving PST had four additional branch scenarios: true positive, true negative, false positive, false negative (Fig 1).
The primary outcome was the incremental cost-effectiveness ratio (ICER) from the health-sector perspective over a 1-year time horizon using quality-adjusted life years (QALYs) as the primary outcome measure for effectiveness. Limited evidence for the indirect cost or health benefits prevented the ability to determine a value from the societal perspective. This study was not considered human research according to the authors’ Institutional Review Board.
All model inputs for the base case scenario along with parameter distribution assumptions are presented in Table 1. Clinical effectiveness and mortality inputs focused on the probability of antibiotic treatment success for MSSA bacteremia in patients admitted with a reported PCN allergy with no confirmatory testing and with PST-guided therapy. The PST-guided therapy assumptions in the acute setting were based on treatment with cefazolin versus vancomycin as described by Blumenthal et al. Outpatient antibiotic estimates were based off of outpatient studies by Sade et al. and Macy.[15, 16] Patients in PST-guided therapy branches included probabilities of skin test errors based on previously reported sensitivity and specificity of the procedure. For patients with no confirmatory PST testing, it was assumed that “self-reported” allergies guided the treatment decision between cefazolin and vancomycin, such that 100% of the patients would have received vancomycin. Treatment success was also defined with and without adverse reactions to the antibiotic selected. Readmission rates were estimated from the probability of MSSA recurrence with vancomycin and cefazolin.
All patients were assigned a baseline disease state QALY value for a post-septic episode. In order to account for health related quality of life for patients experiencing an MSSA recurrence or adverse event to therapy, we applied a discounted QALY value estimated from previous studies in bacteremia.[20–23] Death was assigned a utility score of zero and no patients could experience a negative QALY.
Costs in each scenario were assigned based on disease state assumptions. All patients receiving PST would incur additional costs of the test itself and an approximation of labor and ancillary supplies to administer. Costs of skin testing supplies are approximately $150 per patient. Labor and ancillary supplies to administer may vary from institution to institution based on the site’s established protocol.[13, 24–26] For this study, the base case PST cost would be $300 per patient and increased to test model sensitivity to PST implementation cost increases. Patients who had a negative PST would experience the lower costs of antibiotic treatment estimated for non-PCN-allergy during the initial inpatient encounter and in the outpatient setting post-discharge based on previously published cost estimates for antibiotic regimens in each setting for patients with and without PCN-allergy.[15, 25, 27] Patients with a positive PST and patients not receiving PST would incur costs as a PCN-allergic patient during the inpatient stay and in the outpatient setting post-discharge estimated over 1 year.[15, 25] All patients would experience the direct medical costs for the initial MSSA bacteremia admission and post-discharge outpatient clinic costs related to the disease. Disease recurrence would add the costs of a second hospitalization for MSSA bacteremia and post-discharge costs. The cost of an adverse reaction to drug treatment was estimated from the cost of anaphylaxis and applied to the initial inpatient encounter.[29, 30]
Initial one-way sensitivity was conducted by varying the cost of providing the PST service to estimate a break-even point for both scenarios. A probabilistic sensitivity analysis (PSA) was conducted by assigning distributions for clinical effectiveness, costs for each health state, and utility adjustments for each health state (Table 1). Input variability was based on published evidence where available. We applied a 10% standard deviation estimate across clinical effectiveness and utility parameters and a 20% standard deviation across cost estimates to account for the greater uncertainty with limited evidence for costs in MSSA bacteremia. The PSA used a Monte Carlo simulation of 1,000 repetitions of the model using the parameter distributions in Table 1.
Over a 1-year time horizon, PST services applied to all MSSA bacteremia patients reporting a PCN-allergy would result in a total cost per patient of $12,559 and 0.73 QALYs while no PST services would have a higher cost per patient of $13,219 and 0.66 QALYs per patient (Table 2). The resulting incremental savings of PST services per patient was $660 with an additional 0.07 QALY gained compared with no confirmatory testing. This resulted in a cost-effectiveness estimate of -$9,429 per QALY gained.
Varying the cost of implementing PST services determined a break-even point of $959.98 where any PST cost less than this amount would actually be cost-saving and thereby dominating the no-PST decision. 95% of the incremental cost calculations in the PSA fell between [-$5,141, $3,476] and 95% of the incremental effects fell between [-0.187 QALYs, 0.319 QALYs]. Assuming a willingness-to-pay threshold of $50,000/QALY, 84% of the iterations would have determined PST to be cost-effective. At a $100,000/QALY threshold, 78% of the iterations would have determined PST was cost-effective. QALY gains were observed in 72% of the PSA iterations. In 60% of the PSA iterations, the use of PST was cost-saving when the willingness-to-pay for QALY gains was set at $0.
The cost-effectiveness estimate provided in the base-case model of PST services for all self-reported PCN allergic patients with MSSA bacteremia was well below generally accepted cost-effectiveness thresholds and actually estimated to be cost-saving. Considering potential inpatient and outpatient health sector costs over a 1-year time horizon, PST services provide a dominant strategy in that better health outcomes are achieved for less money when compared to treatment decisions assuming the patient has a true PCN allergy. Our one-way sensitivity analysis of PST implementation costs demonstrates that the service would remain dominant up until it reaches a $960 price point. Once the total cost to purchase the supplies and provide the service exceeds $960, the cost-benefit determination would vary based on individual payer willingness-to-pay thresholds for a QALY. This is an important point for institutions evaluating the decision to provide more routine PST services considering the variability of implementation strategies could influence the total cost of implementation.[13, 24–26] Interventions are generally viewed as cost-effective when they fall below thresholds such as $50,000/QALY, $100,000/QALY, or $150,000/QALY. In this case, costs of the intervention may be increased substantially and may still be acceptable to payers based on QALYs gained in this short-term model.
Several studies have suggested de-labeling patients of PCN allergy through confirmatory testing may optimize antibiotic therapy by reducing the use of broad spectrum agents and overall antibiotic costs as many beta-lactams are typically less expensive agents.[13, 16, 25, 34–36] In 2015, Blumenthal et al. simulated clinical outcomes of patients with MSSA bacteremia and a self-reported PCN allergy arguing a full allergy evaluation with skin testing yields the highest rate of clinical cure and lowest MSSA recurrence. This study expands on the model presented by Blumenthal and colleagues to demonstrate the potential financial impact of implementing PST services in this population. Focusing on MSSA bacteremia over all patients receiving an inpatient antibiotic reduces the variability of patients, comorbidities, and treatment options to increase the internal validity of the PST evaluation. Additional research in broader populations may be warranted to determine whether PST services should be offered to any patient presenting with a PCN allergy on admission or whether confirmatory testing should be the standard in outpatient care.
Confirming or removing a PCN allergy label while the patient is admitted helps guide the current treatment decision and has the potential to improve treatment decisions well beyond 1 year. However, we chose to model a single year to aid in payer decision-making as an insurance company may want to estimate the potential impact of paying for a new program in the current year. For patients within one health system, the medical record would reflect the PST results in perpetuity offering potential long-term implications from the one-time test. Providing a patient-friendly pocket card of the PST results may improve allergy history communication across systems but was not considered in this conservative model.
This study was also limited by its focus on the health sector costs. Future studies may consider additional costs important from the patient perspective which may include productivity losses from missed work, transportation to the hospital and clinic, and caregiver burden. Additionally, this study focuses on PST implementation in an acute care setting and does not consider the use of outpatient parenteral antimicrobial therapy or nafcillin use for bacteremia. The costs of providing PST services may be less expensive in the outpatient setting or as a part of more routine care. As an outpatient service, PST could be billed separately rather than being included within a capitated inpatient payment. The costs for patient receiving vancomycin only focused on the acquisition costs of the drugs and did not account for the therapeutic drug monitoring costs frequently observed while a patient is on therapy. Due to the variability of costs based on how drug monitoring practices are implemented, we relied on published cost estimates related antibiotic therapy only. Adding the costs of drug monitoring with vancomycin may increase the overall cost-effectiveness of PST services.
The probability estimates for MSSA treatment success were derived from other models that focused on effectiveness but did not consider costs. Additionally, in real world settings the prescribing physician may go ahead and prescribe cefazolin over vancomycin in cases without the confirmatory testing based on the patient’s history. Accounting for history-guided treatment with cefazolin in the no-PST arm may reduce the incremental cost-effectiveness of an “all PST” versus “no PST” comparison. This may support the argument for challenging PCN-allergic patients with a cephalosporin when risk of reaction is low and saving the PST expense altogether.
As a decision-tree, patient-level factors were not included other than the PCN-allergy label itself. Future analyses may consider different methods (e.g. discrete event simulation) to test the impact of other patient characteristics. The PSA replicated the model 1,000 times with evidence-based variability in the cost and effect inputs which resulted in 16% of the iterations determining PST services were not cost effective at a $50,000/QALY threshold. Decision-makers should be mindful that the overall cost-effectiveness results of PST services in a short-term model are sensitive to the underlying assumptions.
Beta-lactams are the gold-standard treatment selection for MSSA bacteremia, however, patients reporting a PCN allergy may receive sub-optimal alternative therapies such as vancomycin. The model conducted in this study identified that an inpatient PST service is cost-effective for patients with MSSA bacteremia.
- 1. Mostofsky E, Lipsitch M, Regev-yochay G (2011) Is methicillin-resistant Staphylococcus aureus replacing methicillin-susceptible S. aureus? J Antimicrob Chemother 66:2199–2214 pmid:21737459
- 2. Cosgrove SE, Sakoulas G, Perencevich EN, Schwaber MJ, Karchmer AW, Carmeli Y (2003) Comparison of mortality associated with methicillin-resistant and methicillin-susceptible Staphylococcus aureus bacteremia: a meta-analysis. Clin Infect Dis 36:53–9 pmid:12491202
- 3. Stryjewski ME, Szczech LA, Benjamin DK, et al (2007) Use of Vancomycin or First-Generation Cephalosporins for the Treatment of Hemodialysis-Dependent Patients with Methicillin-Susceptible Staphylococcus aureus Bacteremia. Clin Infect Dis 44:190–196 pmid:17173215
- 4. Wong D, Wong T, Romney M, Leung V (2016) Comparative effectiveness of β-lactam versus vancomycin empiric therapy in patients with methicillin-susceptible Staphylococcus aureus (MSSA) bacteremia. Ann Clin Microbiol Antimicrob. pmid:27112143
- 5. Schweizer ML, Furuno JP, Harris AD, Johnson JK, Shardell MD, McGregor JC, Thom KA, Cosgrove SE, Sakoulas G, Perencevich EN (2011) Comparative effectiveness of nafcillin or cefazolin versus vancomycin in methicillin-susceptible Staphylococcus aureus bacteremia. BMC Infect Dis. pmid:22011388
- 6. Kim SH, Kim KH, Kim H Bin, Kim NJ, Kim EC, Oh MD, Choe KW (2008) Outcome of vancomycin treatment in patients with methicillin-susceptible Staphylococcus aureus bacteremia. Antimicrob Agents Chemother 52:192–197 pmid:17984229
- 7. McDanel JS, Perencevich EN, Diekema DJ, Herwaldt LA, Smith TC, Chrischilles EA, Dawson JD, Jiang L, Goto M, Schweizer ML (2015) Comparative Effectiveness of Beta-Lactams Versus Vancomycin for Treatment of Methicillin-Susceptible Staphylococcus aureus Bloodstream Infections among 122 Hospitals. Clin Infect Dis 61:361–367 pmid:25900170
- 8. Macy E (2015) Penicillin allergy: Optimizing diagnostic protocols, public health implications, and future research needs. Curr Opin Allergy Clin Immunol 15:308–313 pmid:26110680
- 9. PRE-PEN [Package Insert].
- 10. Sogn DD, Evans R, Shepherd GM, et al (1992) Results of the National Institute of Allergy and Infectious Diseases Collaborative Clinical Trial to Test the Predictive Value of Skin Testing With Major and Minor Penicillin Derivatives in Hospitalized Adults. Arch Intern Med 152:1025–1032 pmid:1580706
- 11. Gadde J, Spence M, Wheeler B, Adkinson NF (1993) Clinical Experience With Penicillin Skin Testing in a Large Inner-City STD Clinic. JAMA J Am Med Assoc 270:2456–2463
- 12. Abbo LM, Beekmann SE, Hooton TM, Johannsson B, Polgreen PM (2013) Management of antimicrobial allergies by infectious diseases physicians. JAMA Intern Med 173:1376–1378
- 13. Heil EL, Bork JT, Schmalzle SA, Kleinberg M, Kewalramani A, Gilliam BL, Buchwald UK (2016) Implementation of an infectious disease fellow-managed penicillin allergy skin testing service. Open Forum Infect Dis 3:1–7
- 14. Blumenthal KG, Parker RA, Shenoy ES, Walensky RP (2015) Improving clinical outcomes in patients with methicillin-sensitive staphylococcus aureus bacteremia and reported penicillin allergy. Clin Infect Dis 61:741–749 pmid:25991471
- 15. Sade K, Holtzer I, Levo Y et al. (2003) The economic burden of antibiotic treatment of penicillin- allergic patients in internal medicine wards of a general tertiary care hospital. Clin Exp Allergy 33:501–506 pmid:12680867
- 16. Macy E (1998) Elective penicillin skin testing and amoxicillin challenge: effect on outpatient antibiotic use, cost, and clinical outcomes. J Allergy Clin Immunol 102:281–5 pmid:9723673
- 17. Salkind AR, Cuddy PG, Foxworth JW (2001) Is This Patient Allergic to Penicillin? An Evidence-Based Analysis of the Likelihood of Penicillin Allergy. JAMA 285:2498–2505 pmid:11368703
- 18. Tice AD, Rehm SJ, Dalovisio JR, Bradley JS, Martinelli LP, Graham DR, Gainer RB, Kunkel MJ, Yancey RW, Williams DN (2004) Practice Guidelines for Outpatient Parenteral Antimicrobial Therapy. Clin Infect Dis 38:1651–1671 pmid:15227610
- 19. McComb MN, Collins CD (2014) Comparative cost-effectiveness of alternative empiric antimicrobial treatment options for suspected enterococcal bacteremia. Pharmacotherapy 34:537–544 pmid:24390863
- 20. Bennett JE, Sumner W, Downs SM, Jaffe DM (2000) Parents’ utilities for outcomes of occult bacteremia. Arch Pediatr Adolesc Med 154:43–48 pmid:10632249
- 21. Shiragami M, Mizukami A, Leeuwenkamp O, Mrkvan T, Delgleize E, Kurono Y, Iwata S (2015) Cost-Effectiveness Evaluation of the 10-Valent Pneumococcal Non-typeable Haemophilus influenzae Protein D Conjugate Vaccine and 13-Valent Pneumococcal Vaccine in Japanese Children. Infect Dis Ther 4:93–112
- 22. Kamiya H, Cho BH, Messonnier ML, Clark TA, Liang JL (2016) Impact and cost-effectiveness of a second tetanus toxoid, reduced diphtheria toxoid, and acellular pertussis (Tdap) vaccine dose to prevent pertussis in the United States. Vaccine 34:1832–1838 pmid:26899377
- 23. Lee GM, Salomon JA, LeBaron CW, Lieu TA (2005) Health-state valuations for pertussis: Methods for valuing short-term health states. Health Qual Life Outcomes 3:1–14
- 24. Narayanan PP, Jeffres MN (2017) Feasibility, Benefits, and Limitations of a Penicillin Allergy Skin Testing Service. Ann Pharmacother 51:504–510 pmid:28152605
- 25. Jones BM, Bland CM (2017) Penicillin skin testing as an antimicrobial stewardship initiative. Am J Heal Pharm 74:232–237
- 26. Wall GC, Peters L, Leaders CB, Wille JA (2004) Pharmacist-mandged service providing penicillin allergy skin tests. Am J Heal Pharm 61:1271–5
- 27. Mattingly TJ, Fulton A, Lumish RA, Williams AMC, Yoon SJ, Yuen M, Heil EL (2018) The Cost of Self-Reported Penicillin Allergy: A Systematic Review. J Allergy Clin Immunol Pract. pmid:29355644
- 28. Filice GA, Nyman JA, Lexau C, Lees CH, Bockstedt LA, Como-Sabetti K, Lesher LJ, Lynfield R (2010) Excess Costs and Utilization Associated with Methicillin Resistance for Patients with Staphylococcus aureus Infection. Infect Control Hosp Epidemiol 31:365–373 pmid:20184420
- 29. Slover JD, Phillips MS, Iorio R, Bosco J (2015) Is routine antibiotic prophylaxis cost effective for total joint replacement patients? J Arthroplasty 30:543–6 pmid:25483838
- 30. Flabbee J, Petit N, Jay N, Guénard L, Codreanu F, Mazeyrat R, Kanny G, Moneret-Vautrin DA (2008) The economic costs of severe anaphylaxis in France: An inquiry carried out by the Allergy Vigilance Network. Allergy Eur J Allergy Clin Immunol 63:360–365
- 31. Caro JJ, Briggs AH, Siebert U, Kuntz KM, ISPOR-SMDM Modeling Good Research Practices Task Force (2012) Modeling good research practices—overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force—1. Med Decis Making 15:796–803
- 32. Neumann PJ, Cohen JT, Weinstein MC (2014) Updating Cost-Effectiveness—The Curious Resilience of the $50,000-per-QALY Threshold. N Engl J Med 371:796–797 pmid:25162885
- 33. Neumann PJ, Willke RJ, Garrison LP (2018) A Health Economics Approach to US Value Assessment Frameworks—Introduction: An ISPOR Special Task Force Report . Value Heal 21:119–123
- 34. King EA, Challa S, Curtin P, Bielory L (2016) Penicillin skin testing in hospitalized patients with β-lactam allergies. Effect on antibiotic selection and cost. Ann Allergy, Asthma Immunol 117:67–71
- 35. Li M, Krishna MT, Razaq S, Pillay D (2014) A real-time prospective evaluation of clinical pharmaco-economic impact of diagnostic label of “penicillin allergy” in a UK teaching hospital. J Clin Pathol 67:1088–1092 pmid:25185139
- 36. Satta G, Hill V, Lanzman M, Balakrishnan I (2013) '-Lactam Allergy: Clinical Implications and Costs. Clin Mol allergy 11:2 pmid:24283514
- 37. Sanders GD, Neumann PJ, Basu A, et al (2016) Recommendations for Conduct, Methodological Practices, and Reporting of Cost-effectiveness Analyses. JAMA 316:1093 pmid:27623463