Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Appropriate Use of Cardiac Stress Testing with Imaging: A Systematic Review and Meta-Analysis

  • Joseph A. Ladapo ,

    JLadapo@mednet.ucla.edu

    Affiliation Division of General Internal Medicine and Health Services Research, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA, United States of America

  • Saul Blecker,

    Affiliation Department of Population Health and Medicine, New York University School of Medicine, New York, NY, United States of America

  • Michael O'Donnell,

    Affiliation New York University School of Medicine, New York, NY, United States of America

  • Saahil A. Jumkhawala,

    Affiliation New York University, New York, NY, United States of America

  • Pamela S. Douglas

    Affiliation Department of Medicine, Duke University School of Medicine, Durham, NC, United States of America

Appropriate Use of Cardiac Stress Testing with Imaging: A Systematic Review and Meta-Analysis

  • Joseph A. Ladapo, 
  • Saul Blecker, 
  • Michael O'Donnell, 
  • Saahil A. Jumkhawala, 
  • Pamela S. Douglas
PLOS
x

Abstract

Background

Appropriate use criteria (AUC) for cardiac stress tests address concerns about utilization growth and patient safety. We systematically reviewed studies of appropriateness, including within physician specialties; evaluated trends over time and in response to AUC updates; and characterized leading indications for inappropriate/rarely appropriate testing.

Methods

We searched PubMed (2005–2015) for English-language articles reporting stress echocardiography or myocardial perfusion imaging (MPI) appropriateness. Data were pooled using random-effects meta-analysis and meta-regression.

Results

Thirty-four publications of 41,578 patients were included, primarily from academic centers. Stress echocardiography appropriate testing rates were 53.0% (95% CI, 45.3%–60.7%) and 50.9% (42.6%–59.2%) and inappropriate/rarely appropriate rates were 19.1% (11.4%–26.8%) and 28.4% (23.9%–32.8%) using 2008 and 2011 AUC, respectively. Stress MPI appropriate testing rates were 71.1% (64.5%–77.7%) and 72.0% (67.6%–76.3%) and inappropriate/rarely appropriate rates were 10.7% (7.2%–14.2%) and 15.7% (12.4%–19.1%) using 2005 and 2009 AUC, respectively. There was no significant temporal trend toward rising rates of appropriateness for stress echocardiography or MPI. Unclassified stress echocardiograms fell by 79% (p = 0.04) with updated AUC. There were no differences between cardiac specialists and internists.

Conclusions

Rates of appropriate use tend to be lower for stress echocardiography compared to MPI, and updated AUC reduced unclassified stress echocardiograms. There is no conclusive evidence that AUC improved appropriate use over time. Further research is needed to determine if integration of appropriateness guidelines in academic and community settings is an effective approach to optimizing inappropriate/rarely appropriate use of stress testing and its associated costs and patient harms.

Introduction

Cardiac imaging has advanced physicians’ ability to diagnose and treat a variety of diseases, but rapid growth in the utilization and cost of imaging technology has spurred public and private insurers to scrutinize its use and construct policies aimed at reducing imaging expenditures.[13] Professional society organizations and clinical researchers have also taken steps to better characterize the value of cardiac imaging,[46] while also highlighting clinical scenarios under which imaging use is particularly low-value and unlikely to improve patients’ health or management. While the Choosing Wisely campaign is perhaps the most widely recognized of these professional efforts to self-regulate use of low-value tests and procedures, it was preceded and informed, in part, by the American College of Cardiology’s (ACC) development of appropriate use criteria (AUC) for cardiac imaging stress tests.[7] These AUC have expanded to inform the use of a variety of imaging studies and invasive procedures, but cardiac stress testing has been a focal point of attention, largely due to its wide dissemination,[2] radiation risks,[8] procedural risks, expense, and association with downstream testing and procedures—some of which are invasive.[9] However, until recently, little was known about the potential long-term impact of the ACC’s appropriate use criteria on clinical decision-making in patients evaluated for ischemic heart disease.[10]

We aimed to (1) systematically review studies of cardiac stress testing appropriateness, including appropriateness within physician specialties; (2) evaluate trends over time and in response to updates of AUC; and (3) characterize leading indications for inappropriate/rarely appropriate testing.

While a recent meta-analysis provided important insights into trends in appropriateness across several cardiac imaging modalities,[10] our study differs from this prior work in important ways: we include a greater number of published studies, report a wider range of information about patients characteristics in each study, provide information about indications for inappropriate/rarely appropriate testing, perform more robust analyses of appropriateness by physician specialty (we use both meta-regression and meta-analysis to compare cardiac specialists and internists), and apply a more rigorous method for evaluating temporal trends (we pooled more studies and adjusted for AUC version). Simply stated, we add a more methodologically rigorous meta-analysis to the literature on cardiac imaging appropriateness.

Methods

Search Strategy

We searched PubMed (which includes the MEDLINE database and other sources) from October 1, 2005 to March 1, 2015 for English-language articles reporting stress echocardiography and radionuclide myocardial perfusion imaging (MPI) appropriateness. Our search terms included the Medical Subject Headings exercise test, Cardiac Imaging Techniques, myocardial perfusion imaging, single photon emission computed tomography, and echocardiography; keywords identifying cardiac imaging stress tests, including stress test, thallium, sestamibi, Technetium, myocardial perfusion, MPI, SPECT, and echo; and keywords identifying appropriateness evaluations, including approp* (for “appropriate” and variants), and inapprop* (for “inappropriate” and variants). We identified additional publications through discussion between collaborators. Our report adheres to guidelines for systematic reviews recommended by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement and Metaanalysis of Observational Studies in Epidemiology (MOOSE) group (see Supplemental materials).

Study Selection

Two investigators (J.L. and S.B.), working independently, in duplicate, identified studies eligible for further review after screening titles or abstracts. Studies then underwent full-text retrieval and data extraction if authors reported rates of appropriate or inappropriate cardiac stress testing based on published AUC. Studies were ineligible for inclusion if they focused on special populations (e.g., transplant candidates) whose clinical characteristics made them less representative of general populations undergoing cardiac stress testing, though we did include one study that enrolled only patients with acute chest pain.[11] When multiple studies reported appropriateness outcomes on identical or overlapping populations, only studies that reported unique outcomes were included (see Table 1 footnote for more details). When a cohort was evaluated with the original and updated AUC, both cohorts were included in the meta-analysis, but in separate strata. However, for meta-regression models, only the cohort enrolled in the year closest to the publication date of the AUC was included.

Data Extraction

Using a standardized protocol and reporting form, data were extracted on the following characteristics: (1) identifying information (first author, journal, country, institution, publication year); (2) AUC used (stress echocardiography 2008 or 2011 AUC, stress MPI 2005 or 2009 AUC); (3) patient characteristics (mean age, percentage of male patients, percentage of patients with a history of diabetes, hypertension, hyperlipidemia, body mass index>30, myocardial infarction (MI), percutaneous transluminal coronary angioplasty (PTCA), or coronary artery bypass grafting (CABG); (4) stress test characteristics (test used, type of stressor); (5) appropriateness patterns, including appropriateness stratified by physician specialty; and (6) indications for inappropriate/rarely appropriate testing. We recalculated appropriateness rates when authors excluded patients whose studies were unclassified, but did not include papers that did not report the number of patients who were unclassified. Disagreements between reviewers were resolved through discussion.

Statistical Analysis

The primary outcomes were the proportions of appropriate, inappropriate/rarely appropriate, uncertain/may be appropriate, and unclassified cardiac imaging stress tests. Patient characteristics were summarized after weighting by each study’s sample size. When studies reported that no patients were categorized as unclassified, a 0.5 correction factor was added to that outcome to facilitate calculation of a rate and standard error. Appropriateness estimates were pooled using the DerSimonian—Laird random-effects model to account for between-study heterogeneity attributable to differences in patient populations and clinician practice patterns. Statistical heterogeneity was also assessed with the Cochran Q statistic (a weighted sum of squared differences between studies with a χ2 distribution) and I2 statistic, which is derived from the Q statistic ([Q − df/Q] x 100) and estimates the proportion of overall variation attributable to between-study heterogeneity rather than chance. Because rates of uncertain/may be appropriate and unclassified patients tended to be low, we log transformed these values to more accurately estimate their standard errors and confidence intervals. To assess for publication bias, we constructed funnel plots (standard error versus appropriateness rates) stratified by AUC and performed the Egger test when at least 10 studies were present. None of these plots or statistical tests raised concerns for publication bias.

Meta-regression for temporal trends and effects of AUC updates

We performed meta-regression to assess temporal trends in appropriate and inappropriate/rarely appropriate cardiac stress testing. Meta-regression in this context is limited by the possibility of ecological bias (sometimes referred to as “aggregation bias” or “ecological confounding”),[12] since appropriateness rates in different cohorts over time may not reflect overall trends in appropriateness. We hypothesized that academic setting, prevalence of risk factors for ischemic heart disease (gender, age, comorbidities), and physician specialty would influence rates of appropriateness. However, because many studies reported only a few risk factors, we limited our patient covariates to gender and age, so as not to significantly reduce sample size for these regression models.[12] Separate models pooled all stress echocardiography or MPI studies, and we included an indicator for the specific AUC used. The key variable in these models was time, as captured by the midpoint of the enrollment period. To avoid double-counting, when the same stress echocardiography or MPI cohort was evaluated with original and updated AUC, we used the AUC whose publication date was closest to the enrollment dates.

We also used meta-regression to examine whether updated stress echocardiography and MPI AUC were associated with a reduction in unclassified patients, and to test whether cardiac specialists (cardiologists and cardiac surgeons) and internists had different rates of appropriate and inappropriate cardiac stress testing. Most studies reporting specialty appropriateness categorized physicians as cardiac specialists or non-cardiac specialists, but we considered the latter to be internists based on national referral patterns.[1] These regression models included indicators for the AUC version and presence of cardiac specialists (physician specialty model). We also explored performing a comparison of the pre-2005 period to the post-2005 period but were unable to do so because patient enrollment for all studies included in our meta-analysis began during or after 2005, with the exception of Cortigiani et al 2012. A 2-tailed P-value of <0.05 was considered statistically significant. Analyses were performed in Stata (version 14, StataCorp, College Station, Texas) with the metan family of functions.

Results

Literature Search

Our literature search yielded a total of 3,244 citations, of which 3,122 were excluded after initial screening of abstracts or titles (Fig 1). Of the remaining 122 citations, 34 met inclusion criteria and were selected for full-text review and data extraction. These articles included 6 articles and 6 cohorts for the stress echocardiography 2008 AUC,[11, 1317] 6 articles and 8 cohorts for the stress echocardiography 2011 AUC,[13, 1721] 10 articles and 11 cohorts for the stress MPI 2005 AUC,[2231] and 18 articles and 21 cohorts for the stress MPI 2009 AUC.[14, 20, 23, 30, 3245] Some studies contributed multiple cohorts to our meta-analysis. For example, for stress echocardiography 2011 AUC, Willens et al contributed three cohorts (three separate cohorts that underwent testing in August-September 2008, July-September 2011, and October-December 2011).[17] Similarly, for stress MPI 2005 AUC, Soine et al contributed two cohorts (one cohort underwent testing at University of Washington Medical Center and the second cohort underwent testing at the Veterans Health Administration of Puget Sound).[31] Each cohort is separately presented in the figures (note that enrollment dates are rounded to the nearest year).

The characteristics of these studies and their 41,578 participants are shown in Table 1. The mean age was 63.3 years, 40.4% were women, 12.7% had a prior history of myocardial infarction (reported in 16 studies), and 22.3% had a prior history of revascularization (reported in 18 studies). Overall, population characteristics were generally similar across studies.

Rates of appropriate and inappropriate testing

Appropriate cardiac stress testing rates were 53.0% (95% CI, 45.3% to 60.7%) and 50.9% (95% CI, 42.6% to 59.2%) with stress echocardiography 2008 and 2011 AUC, and 71.1% (95% CI, 64.5% to 77.7%) and 72.0% (95% CI, 67.6% to 76.3%) with stress MPI 2005 and 2009 AUC, respectively (Figs 2 and 3). Inappropriate/rarely appropriate cardiac stress testing rates were 19.1% (95% CI, 11.4% to 26.8%) and 28.4% (95% CI, 23.9% to 32.8%) with stress echocardiography 2008 and 2011 AUC, and 10.7% (95% CI, 7.2% to 14.2%), and 15.7% (95% CI, 12.4% to 19.1%) with stress MPI 2005 and 2009 AUC (Figs 4 and 5).

thumbnail
Fig 2. Appropriate Use Rates of Stress Echocardiography and MPI, Sorted by Patient Enrollment Year.

https://doi.org/10.1371/journal.pone.0161153.g002

thumbnail
Fig 3. Inappropriate/Rarely Appropriate Use Rates of Stress Echocardiography and MPI, Sorted by Patient Enrollment Year.

https://doi.org/10.1371/journal.pone.0161153.g003

thumbnail
Fig 4. Uncertain/May be Appropriate Use Rates of Stress Echocardiography and MPI, Sorted by Patient Enrollment Year.

https://doi.org/10.1371/journal.pone.0161153.g004

thumbnail
Fig 5. Unclassified Use Rates of Stress Echocardiography and MPI, Sorted by Patient Enrollment Year.

https://doi.org/10.1371/journal.pone.0161153.g005

Temporal trends in appropriate and inappropriate/rarely appropriate testing

We examined temporal trends and separately pooled all stress echocardiography or MPI studies, while controlling for AUC version, academic setting, population age, and population gender. For stress echocardiography and MPI, the average annual changes in appropriate testing were -1.9% (95% CI, -4.6% to 0.8%; adjusted change = +1.1%; 95% CI, -11.4% to 13.7%) and +1.9% (95% CI, -0.6% to 4.4%; adjusted change = +1.7%; 95% CI, -1.5% to 4.9%), and the average annual changes in inappropriate/rarely appropriate testing were +0.9% (95% CI, -1.7% to 3.5%; adjusted change = +2.8%; 95% CI, -8.5% to 14.1%) and -0.9% (95% CI, -2.8% to 1.1%; adjusted change = -0.2%; 95% CI, -2.8% to 2.4%), respectively.

In a sensitivity analysis, we attempted to analyze trends in appropriateness within the same institution, but these meta-regression models were not estimable due to limited sample size. However, we provide raw appropriateness rates from these studies: for stress echocardiography, one study from University of Miami Miller School of Medicine with three cohorts reported appropriate rates of 49.8% (2008 AUC, patient recruitment year 2009), 39.2% (2011 AUC, patient recruitment year 2012), and 43.2% (2011 AUC, patient recruitment year 2012). For stress MPI, three studies from Mayo Clinic reported appropriate rates of 64.1% (2005 AUC, patient recruitment year 2005), 66.0% (2005 AUC, patient recruitment year 2007), and 60.1% (2005 AUC, patient recruitment year 2008).[2426]

Uncertain/may be appropriate tests and changes in unclassified tests after AUC updates

Rates of uncertain/may be appropriate and unclassified stress tests for both modalities were generally low, but tended to be higher for stress echocardiography compared to stress MPI. The proportion of testing that was considered uncertain/may be appropriate was 5.7% (95% CI, 3.4% to 9.5%) and 13.9% (95% CI, 9.2% to 20.3%) with stress echocardiography 2008 and 2011 AUC, and 11.6% (95% CI, 9.6% to 14.1%) and 8.2% (95% CI, 6.5% to 10.5%) with stress MPI 2005 and 2009 AUC, respectively. The proportion of testing that was unclassified was 19.4 (95% CI, 11.4% to 33.0%) and 2.2% (95% CI, 0.9% to 5.6%) with stress echocardiography 2008 and 2011 AUC, and 5.4% (95% CI, 3.5% to 8.2%) and 0.7% (95% CI, 0.3% to 1.5%) with stress MPI 2005 and 2009 AUC, respectively.

A test for differences in unclassified rates demonstrated that the updated stress echocardiography AUC in 2011 was associated with a significant reduction in the proportion of these tests (relative reduction = 79%, p = 0.04). There was no evidence of a reduction in unclassified studies after the updated stress MPI criteria were released in 2009 (relative reduction = 64%, p = 0.25).

Appropriateness by physician specialty

Only 11 studies reported appropriateness rates by physician specialty,[14, 15, 17, 22, 27, 36, 39, 40, 4446] and 3 of these studies focused solely on cardiologists.[14, 17, 36] Pooled appropriateness rates from these specialty studies are reported in Fig 6. A test for heterogeneity demonstrated no significant difference in the proportion of appropriate stress echocardiograms or MPIs ordered by cardiac specialists compared to internists (stress echocardiogram difference = -7.3% [95% CI, -70.7% to 56.2%]; stress MPI difference = +7.5% [95% CI, -9.4% to 24.4%]; both with internists as the reference group), and no significant difference in the proportion of inappropriate/rarely appropriate stress echocardiograms or MPIs (stress echocardiogram difference = +12.5% [95% CI, -45.8% to 70.8%]; stress MPI difference = -10.5% [95% CI, -23.8% to 2.8%]).

Indications for inappropriate testing

Indications for inappropriate/rarely appropriate cardiac stress testing (Table 2) were reported by 7 stress echocardiography studies[1317, 19, 20] and 20 stress MPI studies.[14, 20, 22, 25, 26, 2830, 32, 3442, 44, 45] The three most frequent indications for inappropriate/rarely appropriate testing tended to be preoperative evaluation (range 0.0% to 90.0% for stress echocardiography, 0.0% to 75.0% for stress MPI); evaluation of symptomatic patients (often because they were low risk), had an interpretable electrocardiogram, and could exercise (range 10.5% to 44.4% for stress echocardiography, 5.1% to 57.0% for stress MPI); and evaluation of asymptomatic patients (range 4.0% to 65.0% for stress echocardiography, 0.0% to 60.0% for stress MPI).

thumbnail
Table 2. Indications for Inappropriate/Rarely Appropriate Cardiac Stress Test Use.

https://doi.org/10.1371/journal.pone.0161153.t002

Discussion

By systematically reviewing studies of cardiac stress testing AUC, we found that rates of appropriate use tended to be lower for stress echocardiography compared to stress MPI, and that rates of inappropriate/rarely appropriate use tended to be higher. In the patient recruitment years of 2005 to 2014, we also found that rates of appropriate testing did not change significantly for stress echocardiography or MPI. Importantly, we showed that rates of unclassified stress echocardiograms fell after release of the 2011 AUC, whereas no significant changes were identified after updated stress MPI AUC were released. We did not find differences in appropriateness between physician specialties, though these analyses were substantially limited by sparse reporting. Finally, we found significant variability in indications for inappropriate/rarely appropriate cardiac stress tests, with preoperative testing and testing of low-risk symptomatic or asymptomatic patients representing leading indications.

Our study demonstrates that early efforts of the ACC’s Appropriateness Criteria Working Group have had durable and far-reaching consequences on the trajectory of academic inquiry into appropriate testing, with more than 41 diverse cohorts evaluated since publication of the original 2005 AUC. These evaluations have also extended into the community setting, though academic medical centers remain the dominant site for AUC evaluation. While the rapid growth in cardiac imaging that spurred initial efforts to develop AUC may be slowing—at least for stress MPI—the total number of cardiac stress test referrals in US ambulatory settings has not changed in recent years, and expenditures on inappropriate tests remain substantial.[1]

The main findings of our study are similar to those from a recently published meta-analysis[10] of cardiac imaging appropriateness, but there are important methodological differences: Fonseca et al analyzed 10 stress MPI 2009 articles with 11 cohorts whereas we analyzed 18 stress MPI 2009 with 21 cohorts; we present clinical characteristics from study cohorts that were not presented in Fonseca et al’s work, including the prevalence of diabetes, dyslipidemia, hypertension, smoking, coronary artery disease/myocardial infarction, and obesity; provide information about indications for inappropriate/rarely appropriate testing, which was absent in Fonseca et al; perform more robust analyses of appropriateness by physician specialty (we use both meta-regression and meta-analysis to compare cardiac specialists and internists); and apply a more rigorous method for evaluating temporal trends (we pooled more studies and adjusted for AUC version, whereas Fonseca et al estimated separate models [and therefore had smaller sample sizes] for each AUC version). In the context of AUC design, our study suggests that the potential effects of AUC are unclear, but these analyses are limited by the absence of a control group, and they are vulnerable to ecological bias.[12] We found no conclusive evidence of a trend over time in appropriate or inappropriate/rarely appropriate stress echocardiograms or MPIs. These results are in agreement with the work of Fonseca et al,[10] though our study samples differed (we captured more recently published studies), we used enrollment year instead of publication year as our measure of time, and we included an indicator in our meta-regression models for the AUC version used rather than separately treating studies that used different AUC. Our findings are also similar to the results of another recent meta-analysis that focused on stress MPI.[47] It is important to note, however, that there is substantial uncertainty about the extent to which findings within different cohorts in our meta-analysis reflect general trends.[12] Further, we did not account for geographic variation in appropriate and inappropriate use of cardiac imaging, which may be an important source of confounding.

Notably, a greater number of stress MPI publications reported the results of quality improvement initiatives, such as one study that reported the effects of FOCUS (Formation of Optimal Cardiovascular Utilization Strategies), a Web-based community and quality improvement tool.[43] We hypothesized that higher expenditures on stress MPI and expansion of administrative controls such as prior authorization requirements, in combination with widening public concerns about radiation exposure, may have engendered a climate of urgency in the context of stress MPI. It is also possible that these factors may have had the unintended consequence of causing a shift in ordering practices to stress echocardiography, in order to avoid stress MPI in questionable scenarios or other scenarios. Nonetheless, more concerted efforts to increase appropriate use of stress echocardiography and MPI and reduce inappropriate/rarely appropriate use are needed.

Our findings have important implications for insurers and policymakers. A substantial proportion of cardiac imaging stress tests remain inappropriate/rarely appropriate, and our pooled estimates—based largely on studies from academic medical centers—may underestimate the inappropriate/rarely appropriate use of this technology and overestimate its appropriate use in the community. Notably, some studies, such as Doukky et al, focused on patients undergoing testing in a community setting.[35] These inappropriate/rarely appropriate tests increase healthcare expenditures and are less likely to yield positive findings or improve patients’ health outcomes. It is important to recognize that a goal of zero inappropriate/rarely appropriate use is not only unrealistic but undesirable, as each patient represents unique considerations. While the optimal proportion is unknown, it is likely in the range of 10%, though no formal benchmarks have been proposed. Related to this, the small proportion of unclassified studies and relatively modest proportion of studies with uncertain appropriateness suggest that AUC may be an effective tool for evaluating the value of cardiac imaging stress tests, independent of prior authorization mechanisms and radiology benefits managers. Thus, wider incorporation and application of AUC, particularly in integrated health systems and accountable care organizations, could reduce the need for these alternate methods for constraining unnecessary utilization.

Introduction of the 2013 multimodality AUC adds calcium scoring and nonimaging exercise testing to the cohort of technologies subject to appropriate use review. We attempted to integrate the multimodality AUC into our meta-analysis but no studies rigorously implementing it were available at the time of our literature search. However, we did adopt the terminology of the multimodality AUC (e.g., “rarely appropriate” instead of “inappropriate”) to more closely align our results with current interpretation of appropriateness. Assessing its effects, particularly in cohorts that have previously been evaluated with earlier AUC versions, will provide important insights into the overall effect of multimodality criteria, with possible implications for insurers and policymakers. In the Prospective Multicenter Imaging Study for Evaluation of Chest Pain (PROMISE) trial,[6] all patients had chest pain, shortness of breath, or other symptoms as well as cardiovascular risk factors, and therefore would be considered appropriate candidates for cardiac imaging stress tests by these criteria. However, the routine performance of cardiac stress testing in patients without symptoms remains an important issue.[1]

Our study has several limitations. The majority of AUC evaluations were set in academic medical centers, where clinicians often care for higher-risk patients, may be more aware of AUC, and typically face weaker financial incentives to perform cardiac imaging stress tests. Because of small sample size and sparse reporting, we were unable to include a robust set of covariates in our examination of temporal trends. Moreover, meta-regression has significant limitations, including ecological bias (sometimes referred to as “aggregation bias” or “ecological confounding”),[12] and confounding from omitted variables (such as geographic variation in appropriate and inappropriate use of cardiac imaging, and clinical differences in the patient populations referred for stress echocardiography versus MPI), a risk shared by other analytic models of non-randomized, observational data. Further, the absence of a control group in our study attenuated our ability to causally link temporal changes in appropriateness to AUC development. Other ecological factors, including diffusion of radiology benefit managers and prior authorization programs, reductions in Medicare reimbursement, and the Choosing Wisely campaign, may also have contributed. In addition, application of AUC was not standardized across studies, so use of different methodologies could lead to different conclusions about appropriateness.

Recent AUC versions perform well for definitively categorizing the vast majority of stress echocardiograms and MPIs, but we found no conclusive evidence that diffusion of AUC increased the appropriate use of stress echocardiography or MPI. Overall rates of inappropriate/rarely appropriate testing are relatively low in academic settings, and integration of appropriateness guidelines in both academic and community settings may be an effective approach to further optimizing the inappropriate/rarely appropriate use of cardiac stress testing and its associated costs and patient harms.

Supporting Information

S1 Data. Data File for Meta-analysis.

https://doi.org/10.1371/journal.pone.0161153.s001

(CSV)

S1 Text. PRISMA Checklist.

https://doi.org/10.1371/journal.pone.0161153.s002

(DOCX)

S2 Text. MOOSE Checklist.

https://doi.org/10.1371/journal.pone.0161153.s003

(DOC)

Acknowledgments

Dr. Joseph Ladapo had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Dr. Ladapo's work is supported by a K23 Career Development Award (K23 HL116787) from the National Heart, Lung, and Blood Institute (NHLBI) and he serves as a consultant to CardioDx, Inc. Dr. Blecker’s work is supported by a K08 Career Development Award (K08 HS23683) from the Agency for Healthcare Research and Quality.

Author Contributions

  1. Conceptualization: JL PD.
  2. Data curation: JL SB MO SJ PD.
  3. Formal analysis: JL.
  4. Funding acquisition: JL PD.
  5. Investigation: JL SB MO SJ.
  6. Methodology: JL PD.
  7. Project administration: JL PD.
  8. Resources: JL SB MO SJ PD.
  9. Software: JL.
  10. Supervision: PD.
  11. Validation: JL SB MO SJ PD.
  12. Visualization: JL SB MO SJ PD.
  13. Writing - original draft: JL SB PD.
  14. Writing - review & editing: JL SB MO SJ PD.

References

  1. 1. Ladapo JA, Blecker S, Douglas PS. Physician Decision Making and Trends in the Use of Cardiac Stress Testing in the United States: An Analysis of Repeated Cross-sectional Data. Ann Intern Med. 2014;161(7):482–90. pmid:25285541.
  2. 2. Shaw LJ, Marwick TH, Zoghbi WA, Hundley WG, Kramer CM, Achenbach S, et al. Why all the focus on cardiac imaging? JACC Cardiovasc Imaging. 2010;3(7):789–94. pmid:20633864.
  3. 3. Iglehart JK. Health insurers and medical-imaging policy—a work in progress. N Engl J Med. 2009;360(10):1030–7. pmid:19264694
  4. 4. Ladapo JA, Jaffer FA, Hoffmann U, Thomson CC, Bamberg F, Dec W, et al. Clinical outcomes and cost-effectiveness of coronary computed tomography angiography in the evaluation of patients with chest pain. J Am Coll Cardiol. 2009;54(25):2409–22. Epub 2010/01/20. pmid:20082932.
  5. 5. Shreibati JB, Baker LC, Hlatky MA. Association of coronary CT angiography or stress testing with subsequent utilization and spending among Medicare beneficiaries. JAMA. 2011;306(19):2128–36. Epub 2011/11/18. pmid:22089720.
  6. 6. Douglas PS, Hoffmann U, Patel MR, Mark DB, Al-Khalidi HR, Cavanaugh B, et al. Outcomes of anatomical versus functional testing for coronary artery disease. N Engl J Med. 2015;372(14):1291–300. Epub 2015/03/17. pmid:25773919; PubMed Central PMCID: PMCPMC4473773.
  7. 7. Brindis RG, Douglas PS, Hendel RC, Peterson ED, Wolk MJ, Allen JM, et al. ACCF/ASNC appropriateness criteria for single-photon emission computed tomography myocardial perfusion imaging (SPECT MPI): a report of the American College of Cardiology Foundation Quality Strategic Directions Committee Appropriateness Criteria Working Group and the American Society of Nuclear Cardiology endorsed by the American Heart Association. J Am Coll Cardiol. 2005;46(8):1587–605. Epub 2005/10/18. pmid:16226194.
  8. 8. Brenner DJ. Medical imaging in the 21st century—getting the best bang for the rad. N Engl J Med. 2010;362(10):943–5. Epub 2010/03/12. pmid:20220190.
  9. 9. Miller TD, Roger VL, Hodge DO, Hopfenspirger MR, Bailey KR, Gibbons RJ. Gender differences and temporal trends in clinical characteristics, stress test results and use of invasive procedures in patients undergoing evaluation for coronary artery disease. J Am Coll Cardiol. 2001;38(3):690–7. Epub 2001/08/31. pmid:11527619.
  10. 10. Fonseca R, Negishi K, Otahal P, Marwick TH. Temporal Changes in Appropriateness of Cardiac Imaging. J Am Coll Cardiol. 2015;65(8):763–73. Epub 2015/02/28. pmid:25720619.
  11. 11. Schmitz L, Mori N, Khandheria BK, Gupta A. Appropriateness criteria for stress echocardiography in patients with acute chest pain: are we choosing wisely? Int J Cardiol. 2013;165(2):387–8. Epub 2012/10/02. pmid:23022087.
  12. 12. Thompson SG, Higgins JPT. How should meta-regression analyses be undertaken and interpreted? Stat Med. 2002;21(11):1559–73. pmid:12111920
  13. 13. Bhatia RS, Kumar V, Picard MH, Weiner RB. Comparison of the 2008 and 2011 appropriate use criteria for stress echocardiography. J Am Soc Echocardiogr. 2013;26(4):339–43. Epub 2013/01/15. pmid:23313389.
  14. 14. Lin FY, Dunning AM, Narula J, Shaw LJ, Gransar H, Berman DS, et al. Impact of an automated multimodality point-of-order decision support tool on rates of appropriate testing and clinical decision making for individuals with suspected coronary artery disease: A prospective multicenter study. J Am Coll Cardiol. 2013;62(4):308–16. pmid:23707319
  15. 15. Mansour IN, Lang RM, Aburuwaida WM, Bhave NM, Ward RP. Evaluation of the clinical application of the ACCF/ASE appropriateness criteria for stress echocardiography. J Am Soc Echocardiogr. 2010;23(11):1199–204. Epub 2010/08/21. pmid:20724108.
  16. 16. McCully RB, Pellikka PA, Hodge DO, Araoz PA, Miller TD, Gibbons RJ. Applicability of appropriateness criteria for stress imaging similarities and differences between stress echocardiography and single-photon emission computed tomography myocardial perfusion imaging criteria. Circulation: Cardiovascular Imaging. 2009;2(3):213–8. pmid:19808595
  17. 17. Willens HJ, Nelson K, Hendel RC. Appropriate use criteria for stress echocardiography: impact of updated criteria on appropriateness ratings, correlation with pre-authorization guidelines, and effect of temporal trends and an educational initiative on utilization. JACC Cardiovasc Imaging. 2013;6(3):297–309. Epub 2013/02/26. pmid:23433927.
  18. 18. Bhattacharyya S, Kamperidis V, Chahal N, Shah BN, Roussin I, Li W, et al. Clinical and prognostic value of stress echocardiography appropriateness criteria for evaluation of coronary artery disease in a tertiary referral centre. Heart. 2014;100(5):370–4. Epub 2013/12/07. pmid:24310519.
  19. 19. Cortigiani L, Bigi R, Bovenzi F, Molinaro S, Picano E, Sicari R. Prognostic implication of appropriateness criteria for pharmacologic stress echocardiography performed in an outpatient clinic [corrected]. Circ Cardiovasc Imaging. 2012;5(3):298–305. Epub 2012/04/03. pmid:22467675.
  20. 20. Gertz ZM, O'Donnell W, Raina A, Litwack AJ, Balderston JR, Goldberg LR. Application of appropriate use criteria to cardiac stress testing in the hospital setting: limitations of the criteria and areas for improved practice. Clin Cardiol. 2015;38(1):8–12. Epub 2014/10/23. pmid:25336343.
  21. 21. Mansour IN, Razi RR, Bhave NM, Ward RP. Comparison of the updated 2011 appropriate use criteria for echocardiography to the original criteria for transthoracic, transesophageal, and stress echocardiography. J Am Soc Echocardiogr. 2012;25(11):1153–61. Epub 2012/09/25. pmid:22998855.
  22. 22. Druz RS, Phillips LM, Sharifova G. Clinical evaluation of the appropriateness use criteria for single-photon emission-computed tomography: differences by patient population, physician specialty, and patient outcomes. ISRN cardiology. 2011;2011:798318. Epub 2012/02/22. pmid:22347656; PubMed Central PMCID: PMCPmc3262510.
  23. 23. Gholamrezanezhad A, Shirafkan A, Mirpour S, Rayatnavaz M, Alborzi A, Mogharrabi M, et al. Appropriateness of referrals for single-photon emission computed tomography myocardial perfusion imaging (SPECT-MPI) in a developing community: a comparison between 2005 and 2009 versions of ACCF/ASNC appropriateness criteria. J Nucl Cardiol. 2011;18(6):1044–52. Epub 2011/08/06. pmid:21818700.
  24. 24. Gibbons RJ, Askew JW, Hodge D, Kaping B, Carryer DJ, Miller T. Appropriate use criteria for stress single-photon emission computed tomography sestamibi studies: A quality improvement project. Circulation. 2011;123(5):499–503. pmid:21262995
  25. 25. Gibbons RJ, Askew JW, Hodge D, Miller TD. Temporal trends in compliance with appropriateness criteria for stress single-photon emission computed tomography sestamibi studies in an academic medical center. Am Heart J. 2010;159(3):484–9. pmid:20211313
  26. 26. Gibbons RJ, Miller TD, Hodge D, Urban L, Araoz PA, Pellikka P, et al. Application of appropriateness criteria to stress single-photon emission computed tomography sestamibi studies and stress echocardiograms in an academic medical center. J Am Coll Cardiol. 2008;51(13):1283–9. pmid:18371560.
  27. 27. Gupta A, Tsiaras SV, Dunsiger SI, Tilkemeier PL. Gender disparity and the appropriateness of myocardial perfusion imaging. J Nucl Cardiol. 2011;18(4):588–94. Epub 2011/04/26. pmid:21516377.
  28. 28. Hendel RC, Cerqueira M, Douglas PS, Caruth KC, Allen JM, Jensen NC, et al. A Multicenter Assessment of the Use of Single-Photon Emission Computed Tomography Myocardial Perfusion Imaging With Appropriateness Criteria. J Am Coll Cardiol. 2010;55(2):156–62. pmid:20117384
  29. 29. Mehta R, Ward RP, Chandra S, Agarwal R, Williams KA. Evaluation of the American College of Cardiology Foundation/American Society of Nuclear Cardiology appropriateness criteria for SPECT myocardial perfusion imaging. J Nucl Cardiol. 2008;15(3):337–44. Epub 2008/06/03. pmid:18513640.
  30. 30. Oliveira AD, Rezende MF, Correa R, Mousinho R, Azevedo JC, Miranda SM, et al. Applicability of the Appropriate use Criteria for Myocardial Perfusion Scintigraphy. Arq Bras Cardiol. 2014;103(5):375–81. Epub 2014/09/25. pmid:25252163; PubMed Central PMCID: PMCPmc4262097.
  31. 31. Soine LA, Cunningham SL, Motzer SA, Inoue LY, Caldwell JH. Application of appropriate use criteria for stress myocardial perfusion imaging at two academic medical centers: compliance and association with image findings. J Am Acad Nurse Pract. 2012;24(4):200–8. Epub 2012/04/11. pmid:22486835.
  32. 32. Aldweib N, Negishi K, Seicean S, Jaber WA, Hachamovitch R, Cerqueira M, et al. Appropriate test selection for single-photon emission computed tomography imaging: Association with clinical risk, posttest management, and outcomes. Am Heart J. 2013;166(3):581–8. pmid:24016510
  33. 33. Bohossian HB, Park AW, Holcroft C. The impact of individual variation analysis on myocardial perfusion imaging utilization within a hospitalist group. Journal of hospital medicine. 2015;10(3):190–3. Epub 2014/11/29. pmid:25430810.
  34. 34. Carryer DJ, Hodge DO, Miller TD, Askew JW, Gibbons RJ. Application of appropriateness criteria to stress single photon emission computed tomography sestamibi studies: a comparison of the 2009 revised appropriateness criteria to the 2005 original criteria. Am Heart J. 2010;160(2):244–9. Epub 2010/08/10. pmid:20691828.
  35. 35. Doukky R, Hayes K, Frogge N, Balakrishnan G, Dontaraju VS, Rangel MO, et al. Impact of appropriate use on the prognostic value of single-photon emission computed tomography myocardial perfusion imaging. Circulation. 2013;128(15):1634–43. Epub 2013/09/12. pmid:24021779.
  36. 36. Johnson TV, Rose GA, Fenner DJ, Rozario NL. Improving appropriate use of echocardiography and single-photon emission computed tomographic myocardial perfusion imaging: a continuous quality improvement initiative. J Am Soc Echocardiogr. 2014;27(7):749–57. Epub 2014/04/15. pmid:24726335.
  37. 37. Koh AS, Flores JL, Keng FY, Tan RS, Chua TS. Evaluation of the American College of Cardiology Foundation/American Society of Nuclear Cardiology appropriateness criteria for SPECT myocardial perfusion imaging in an Asian tertiary cardiac center. J Nucl Cardiol. 2011;18(2):324–30. Epub 2010/11/26. pmid:21107927.
  38. 38. Lalude OO, Gutarra MF, Pollono EN, Lee S, Tarwater PM. Inappropriate utilization of SPECT myocardial perfusion imaging on the USA-Mexico border. J Nucl Cardiol. 2014;21(3):544–52. Epub 2014/03/15. pmid:24627346.
  39. 39. Mahajan A, Bal S, Hahn H. Myocardial perfusion imaging determination using an appropriate use smartphone application. J Nucl Cardiol. 2015;22(1):66–71. Epub 2014/10/03. pmid:25273671.
  40. 40. Medolago G, Marcassa C, Alkraisheh A, Campini R, Ghilardi A, Giubbini R. Applicability of the appropriate use criteria for SPECT myocardial perfusion imaging in Italy: preliminary results. Eur J Nucl Med Mol Imaging. 2014;41(9):1695–700. Epub 2014/03/19. pmid:24633473.
  41. 41. Moralidis E, Papadimitriou N, Stathaki M, Xourgia X, Spyridonidis T, Fotopoulos A, et al. A multicenter evaluation of the appropriate use of single-photon emission tomography myocardial perfusion imaging in Greece. J Nucl Cardiol. 2013;20(2):275–83. Epub 2013/02/23. pmid:23430360.
  42. 42. Nelson KH, Willens HJ, Hendel RC. Utilization of radionuclide myocardial perfusion imaging in two health care systems: assessment with the 2009 ACCF/ASNC/AHA appropriateness use criteria. J Nucl Cardiol. 2012;19(1):37–42. Epub 2011/11/03. pmid:22045393.
  43. 43. Saifi S, Taylor AJ, Allen J, Hendel R. The use of a learning community and online evaluation of utilization for SPECT myocardial perfusion imaging. JACC Cardiovasc Imaging. 2013;6(7):823–9. Epub 2013/05/07. pmid:23643281.
  44. 44. Singh M, Babayan Z, Harjai KJ, Dedhia P, Sattur S, Jagasia DH. Utilization patterns of single-photon emission cardiac tomography myocardial perfusion imaging studies in a rural tertiary care setting. Clin Cardiol. 2014;37(2):67–72. Epub 2014/01/09. pmid:24399332.
  45. 45. Winchester DE, Hymas J, Meral R, Nguyen D, Dusaj R, Shaw LJ, et al. Clinician-dependent variations in inappropriate use of myocardial perfusion imaging: training, specialty, and location. J Nucl Cardiol. 2014;21(3):598–604. Epub 2014/03/29. pmid:24671699.
  46. 46. Doukky R, Hayes K, Frogge N. Are cardiologists truly better at appropriately selecting patients for stress myocardial perfusion imaging? Int J Cardiol. 2014;176(1):285–6. Epub 2014/07/22. pmid:25042647.
  47. 47. Elgendy IY, Mahmoud A, Shuster JJ, Doukky R, Winchester DE. Outcomes after inappropriate nuclear myocardial perfusion imaging: A meta-analysis. J Nucl Cardiol. 2015. pmid:26253327.