Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Economic evaluations of eHealth technologies: A systematic review

  • Chiranjeev Sanyal,

    Roles Conceptualization, Data curation, Formal analysis, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation School of Public Health and Health Systems, University of Waterloo, Waterloo, Canada

  • Paul Stolee ,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Methodology, Project administration, Resources, Supervision, Validation, Writing – review & editing

    stolee@uwaterloo.ca

    Affiliation School of Public Health and Health Systems, University of Waterloo, Waterloo, Canada

  • Don Juzwishin,

    Roles Conceptualization, Funding acquisition, Methodology, Project administration, Resources, Supervision, Writing – review & editing

    Affiliation Health Technology Assessment and Innovation, Alberta Health Services, Edmonton, Alberta, Canada

  • Don Husereau

    Roles Conceptualization, Methodology, Validation, Writing – review & editing

    Affiliation Institute of Health Economics, Edmonton, Alberta, Canada

Economic evaluations of eHealth technologies: A systematic review

  • Chiranjeev Sanyal, 
  • Paul Stolee, 
  • Don Juzwishin, 
  • Don Husereau
PLOS
x

Abstract

Background

Innovations in eHealth technologies have the potential to help older adults live independently, maintain their quality of life, and to reduce their health system dependency and health care expenditure. The objective of this study was to systematically review and appraise the quality of cost-effectiveness or utility studies assessing eHealth technologies in study populations involving older adults.

Methods

We systematically searched multiple databases (MEDLINE, EMBASE, CINAHL, NHS EED, and PsycINFO) for peer-reviewed studies published in English from 2000 to 2016 that examined cost-effectiveness (or utility) of eHealth technologies. The reporting quality of included studies was appraised using the Consolidated Health Economic Evaluation Reporting Standards statement.

Results

Eleven full text articles met the inclusion criteria representing public and private health care systems. eHealth technologies evaluated by these studies includes computerized decision support system, a web-based physical activity intervention, internet-delivered cognitive behavioral therapy, telecare, and telehealth. Overall, the reporting quality of the studies included in the review was varied. Most studies demonstrated efficacy and cost-effectiveness of an intervention using a randomized control trial and statistical modeling, respectively. This review found limited information on the feasibility of adopting these technologies based on economic and organizational factors.

Conclusions

This review identified few economic evaluations of eHealth technologies that included older adults. The quality of the current evidence is limited and further research is warranted to clearly demonstrate the long-term cost-effectiveness of eHealth technologies from the health care system and societal perspectives.

Introduction

In developed countries including Canada, life expectancy among individuals 65 years and older has grown due to health technology innovations and advancements in public health among other factors [1, 2]. The population aged 60 and older worldwide is expected to grow from 901 million in 2015 to 1.4 billion in 2030 [1]. The rise of the aging population is associated with increased new cases of cancer, dementia, and mental illness among others [2]. Older adults aged 65–79 and 80+ years have on average 3 or 4 chronic conditions, respectively [2]. The management of new cases and existing conditions will require substantial health care resources [1].

Electronic health (eHealth) technologies utilize information and communication technologies to manage health, deliver care, and manage the health care system [3]. eHealth technologies can play a role in allowing seniors to live at home [3], while increasing the efficiency of the health care system. For example, they may replace face-to-face meetings with health care professionals and provide patient education, counseling services, access to data (and/or collection) and enable health information sharing. In addition, they can facilitate patient monitoring (and support), clinical examination, diagnosis and treatment [3]. These technologies can bring the health care provider and health system to the patient thereby enabling patient-oriented care.

Economic evaluations can inform health care decision makers about efficient allocation of scarce resources to improve health outcomes. Cost-effectiveness (CE) and cost-utility (CU) studies are usually used by decision makers for value proposition of a novel health technology [4]. CE and CU analyses quantify costs and consequences using health outcomes (e.g. life years saved or functional improvement) and quality-adjusted-life-years (QALYs), respectively [4]. In the comparison between two potential health technologies or other interventions, the incremental cost-effectiveness ratio (ICER) quantifies the difference in cost (ΔC) divided by the difference in their effect (ΔE) [4]. An intervention is considered cost-effective if the ICER (ΔC/ΔE) is less than a predetermined maximum amount (λ) the payer is willing to pay (WTP) for a gain in health outcome (i.e. ΔC/ΔE<λ) [4]. Health care decision makers most widely use the ICER to make decisions on adoption and reimbursement of health technologies [4]. The objective of this study was to systematically review and appraise the quality of CE or CU studies assessing eHealth technologies in study populations involving older adults. Moreover, we examined the literature to know whether any conclusion on CE or CU can be made on the use of eHealth technologies in older adults.

Methods

Literature search strategy

This systematic review is reported in accordance with the reporting guidance provided by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement [5]. An experienced health science librarian searched the literature published between January 1st, 2000 to October 4th, 2016 in the following electronic databases: Medline, EMBASE, CINAHL, NHS EED, and PsycINFO. Innovations in eHealth technologies have evolved considerably over the years; therefore, we decided to limit the literature search to articles published in 2000 or later. The search used text words 'assistive technology', 'socially assistive robots', 'mobile health', 'mobile robot', 'smart home system', 'telecare', 'telehealth', 'telemedicine', 'wander prevention systems', 'mobile locator devices', 'gps', 'location based technology', 'mobile apps', 'mobile application', 'cell phone', 'web based', 'internet', 'mhealth', 'm health', ‘eHealth’ or 'e health' cross referenced with 'older adult', 'elderly', 'seniors', or 'older patient' and 'cost effective', 'cost utility', or 'economic evaluation', S1 Text. The reference lists of the included studies were hand searched to identify additional publications.

Selection of studies

This review focussed on studies that conforms to PICO (population, intervention, comparison, and outcomes) criteria older adults, eHealth technologies, standard or usual care and ICER, respectively [6, 7]. Therefore, this review focused on the ICER as the outcome of interest.

Two reviewers reviewed titles, abstracts, and full text articles. Titles and abstracts were screened for relevance based on the research question. Any articles that met all of the inclusion criteria were retained for full text review. The reviewers independently read full text of eligible articles. Disagreements were resolved by consensus between the two reviewers; where they did not reach consensus, this was adjudicated by a third reviewer.

Following inclusion/exclusion criteria were considered to identify relevant studies:

  1. Inclusion criteria -
    1. Peer reviewed studies published in English.
    2. CE or CU studies of eHealth technologies that was conducted alongside a clinical trial or based on simulation modeling.
    3. Study population included older adults that is, individuals on average aged 60 years or older.
  2. Exclusion criteria -
    1. Letters to the editor, conference abstracts, review articles without original data, or grey literature and/or reports.
    2. ICER not reported.
    3. Cost analysis studies (i.e. studies which measured or compared costs without health outcomes).

Data abstraction

Data abstraction for each of the studies in the review included the following information: country, year of publication, intervention, comparator, disease, mean age, sample size, efficacy-effectiveness study design, CE or CU method, perspective, time horizon, year of costing, ICER, and funding source.

Quality assessment

We used the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement to appraise the quality of reporting of the studies [8]. The CHEERS statement includes 24 items that appraises an article on the following criteria: (i) title and abstract, (ii) introduction, (iii) methods, (iv) results, (v) discussion, and (vi) funding and conflict of interest [8].

Results

Overview of studies included

Of the 1474 records identified after removing duplicates, 14 potentially relevant full text articles were reviewed for eligibility, of which 11 studies met our eligibility criteria and were included in the review (Fig 1). The effectiveness (or efficacy) data of two CE studies were based on observational studies and the rest were based on single randomized controlled trials, Table 1. The mean age range of the study population for intervention and control groups were 64.5 to 75.9 and 64.2 to 73.2 years, respectively, Table 1. The sample sizes in the intervention and control group were (n = 24 to 1699) and (n = 21 to 1692), respectively, Table 1.

eHealth technologies evaluated were telehealth [912], a computerized decision support system [13], telemonitoring [14, 15], web-based physical activity intervention [16], telecare [17], and internet delivered cognitive behavior therapy (iCBT) [18, 19] compared with usual (or standard) care in patients with diabetes [9, 13, 16, 17], congestive heart failure [14, 15, 17], cardiovascular disease [11, 12], colon cancer [16], breast cancer [16], acute myocardial infarctions [16], stroke [16], chronic obstructive pulmonary disease [10], generalized anxiety disorder [18], and depression [19]. These studies were conducted from the perspective of public health care system (i.e. The Netherlands [13, 16], Canada [15], United Kingdom [11, 12, 17], Spain [10], and Australia [18, 19]). Two studies used simulation modeling [11, 16], nine studies used statistical (regression) modeling [9, 10, 12, 14, 15, 1719], and one used both simulation and regression modeling [13] to predict costs and QALYs. The time horizon of analysis were 4-months [10], 1-year [9, 12, 14, 15, 1719], 5-years [16], 10-years [16], and lifetime [11, 13, 16]. The choice of time horizon used in the analysis was not justified by these studies. Most of the studies included in the review were funded by public agencies [912, 16, 17], followed by non-profit organisation [18, 19], private [13], and mixed (private and public) [14].

Quality assessment of the included studies

The reporting quality of the studies is presented in Tables 2, S1 and Fig 2 which reflects the strengths and limitations of these studies. The major findings are summarized below.

thumbnail
Fig 2. CHEERS statement quality results.

Items: (1) Title, (2) Abstract, (3) Background and Objectives, (4) Target population and subgroups, (5) Setting and Location, (6) Study perspective, (7) Comparators, (8) Time horizon, (9) Discount rate, (10) Choice of health outcomes, (11) Effectiveness, (12) Preference valuation, (13) Estimate resources and costs, (14) Currency, price date, conversion, (15) Choice of model, (16) Assumptions, (17) Analytical methods, (18) Study parameters, (19) Incremental costs and outcomes, (20) Uncertainty—single study or model based, (21) Heterogeneity, (22) Study findings/limitations/generalizability/current knowledge, (23) Source of funding, (24) Conflict of interest.

https://doi.org/10.1371/journal.pone.0198112.g002

thumbnail
Table 2. Quality assessment of included studies using CHEERS statement.

https://doi.org/10.1371/journal.pone.0198112.t002

Study objective, population characteristics and comparators.

The findings of a CE or CU study may vary based on the characteristics of the population included in a study. Most of the studies included in the review clearly stated their objectives and population characteristics. One study did not clearly state the study objective [16], two studies inadequately described study population characteristics [10, 17]. Four studies did not clearly describe the comparator (e.g. usual care) against which the intervention of interest was compared [1012, 15].

Study perspective.

The perspective of a study guides the cost components and outcomes to be evaluated by a study. Most of these considered the health care perspective; hence, the broader societal perspective was not represented. Two studies did not clearly state the perspective of the study [9, 16]. However, the reporting indicates these studies [9, 16] were most likely conducted from the health care system perspective.

Source of efficacy or effectiveness data.

All included studies were based on single experimental [1019] or non-experimental [9] study. The study design to evaluate the efficacy or effectiveness of these technologies was adequately described within the word limit of the journals. However, these studies did not clearly state why single efficacy or effectiveness study was used to understand whether the best available evidence was considered for the decision problem addressed by the study. Further, experimental designs comprised of randomized control trials (RCTs) [10, 1316, 18, 19] and pragmatic trials [11, 12, 17]. Of note, in contrast to RCTs, pragmatic trials have greater external validity which better reflects real world scenarios.

Preference based health outcomes.

All studies measured QALYs to quantity health outcomes pertaining to the intervention of interest and the comparator. Most studies [912, 14, 15, 1719] reported the preference based instrument (e.g. SF-36 or EQ-5D) used to generate QALYs. In addition, few studies [11, 12, 14, 18, 19] have reported using country specific value sets to reflect the local context. These studies did not clearly state the preference elicitation methods used by the instruments (e.g. standard gamble or time trade off) to realize whether the instrument can adequately represent decision under uncertainty [4].

Measurement of resource use and costing.

Few studies [10, 1416, 18, 19] clearly described health care resource use and cost components considered to address their research question. The year of costing to address inflation was reported by most of the studies [1012, 1417]. These studies did not clearly state whether costing was based on top-down or micro-costing was not clearly stated by these studies [7]. The review identified selective reporting on health care resource use and cost components by the studies.

Methods for base case and uncertainty analysis.

The included studies analyzed patient level data to estimate the ICER. Studies have used either regression [9, 10, 12, 14, 15, 1719], simulation modeling [11, 16] or both [13]. These studies used various regression methods to address specific data requirement (e.g. clustering, correlation between the error terms and costs or QALYs etc.). Two studies [13, 14] reported the influence of patient subgroups on the ICER. These studies used bootstrapping to estimate 95% confidence interval to reflect the uncertainty associated with the estimate.

The model based studies [11, 13, 16] did not clearly describe the methods used to estimate model parameters, list of parameters and state transition probabilities used to develop the model. Schematic representation of the model indicating health state transitions considered and potential transition trajectories were not reported. Inadequate reporting was observed with regard to uncertainty analysis (deterministic or probabilistic) [11, 13, 16]. For probabilistic sensitivity analysis the parameter distributions, ranges etc. used to assess uncertainty was not reported [11, 13, 16]. The uncertainty with the data was reflected by 95% confidence interval. Cost effectiveness acceptability curves were reported to highlight the probability of the eHealth technology being CE at varying thresholds of willingness-to-pay [11, 16].

Discussion

CE or CU studies are important to assess the value for money of novel health technologies and are widely used for the adoption and funding decisions by governments, stakeholders, insurers, and health policy makers [4, 7]. This study sought to systematically review and critically appraise the existing literature on CE or CU of eHealth technologies involving older adults. Eleven studies included in the review evaluated various eHealth technologies and comparators in the management of chronic diseases, using different outcomes. The CHEERS statement [8] helped assess the quality of reporting of these studies.

Most of the studies included in this review used the clinical evidence generated by a single RCT. Therefore, generalizability of findings is limited since RCT’s are controlled studies and may not represent routine clinical practice well. The majority of the studies lacked detailed description on unit costs, data sources, and cost calculations. The methodology used to calculate costs can significantly influence the overall cost estimates. It is possible the word limit of journals may prevent authors from providing detailed description of health services resource use, corresponding unit cost, and cost calculation. However, reporting these details can help decision makers understand what costs were considered for the analyses and the extent to which they are pertinent to their settings. Most of the studies used a time horizon of 1-year and failed to assess long-term costs and QALYs. Further, the rationale for the choice of time horizon for the analyses was not stated by these studies. The choice of time frame may influence study findings. Given that these studies were conducted in patients with chronic diseases, a longer time horizon will be preferred to adequately represent long term health services use, costs, and QALYs.

Studies identified in this review were conducted in various countries, in the context of those countries’ health care systems. Applying results from differing contexts is difficult due to variations in clinical practice, unit costs, health care delivery, and perspective of analysis, among others. Consistent across studies was a lack of reporting on the feasibility of adopting these technologies based on economic and organizational factors. For example, whether adopting these technologies will lead to an increase in health care spending or in resources for staff training and change management (among others) was not discussed. Such insight will help decision makers to decide upon various scenarios to consider while adopting and implementing the novel technology into their specific context.

There are limitations associated with this systematic review. First, peer-reviewed studies published in English were considered for this study. Therefore, studies published in other languages were excluded by our search strategy. Second, our eligibility criteria was met by eleven studies that evaluated a range of eHealth technologies in the management of chronic diseases. Therefore, we were not able to categorize eHealth technologies by the type of technology or disease management. Moreover, included studies were of varying methodological rigor making it difficult to compare study findings. Third, this review included CE or CU studies of eHealth technologies that included older adults. Therefore, inclusion of other age groups can potentially result in more articles. Finally, we did not review the grey literature.

In light of the limitations of the existing evidence, more research on the cost-effectiveness of eHealth technologies is warranted. Future studies can potentially consider the following: conform to reporting statement [8] to demonstrate methodological rigor, meta-analysis (data permitting) to synthesize the clinical evidence or use real world data, descriptions of valuation of health services use and corresponding unit costs and costing methodology, adequate characterization of uncertainty with study findings using deterministic or probabilistic analysis, analysis from different perspectives, and use of longer time horizons accounting for various pathways used in routine disease management [20]. In order to support healthy aging in place, it will also be highly relevant to evaluate the cost-effectiveness of using eHealth technologies to deliver integrated care in older adults affected by Alzheimer's disease, arthritis, and other chronic diseases. Future studies may extend their analysis beyond “single-point-in-time” [21] technology adoption decision making [20, 21] and address questions from various perspectives (stakeholder, patient, public, health system, payer and/or industry) based on the context [22], continuum of care and life cycle evaluation of a health technology [20].

Conclusions

eHealth technologies can be used to provide resource efficient patient-oriented care. This review identified growing use of these technologies in the management of chronic diseases in study populations including older adults. Given the limitations of these studies, there is a lack of convincing evidence to conclude whether the use of eHealth technologies to deliver health care to older adults will demonstrate value at any acceptable level of investment. It is important to improve the methodological rigour and reporting of CE or CU studies so that decisions to use eHealth technologies are informed by convincing evidence of a value proposition. In addition, studies should evaluate the long-term clinical and cost-effectiveness of these technologies along the continuum of care from health care and/or societal perspectives.

Supporting information

Acknowledgments

This work is funded by the AGE-WELL Network of Centres of Excellence (NCE), Canada including a Post-Doctoral Fellowship awarded to Chiranjeev Sanyal.

We are grateful to Jackie Stapleton, Health Sciences librarian, for her expertise and assistance in designing and conducting the literature searches for this review.

References

  1. 1. United Nations, Department of Economic and Social Affairs, Population Division (2015). World Population Ageing 2015 (ST/ESA/SER.A/390).
  2. 2. The State of Seniors Health Care in Canada. Available at https://www.cma.ca/En/Lists/Medias/the-state-of-seniors-health-care-in-canada-september-2016.pdf. Accessed on October 31, 2016.
  3. 3. Alvarez RC: The promise of e-Health—a Canadian perspective. eHealth Int 2002 Sep 17;1(1):4. pmid:12459044
  4. 4. Drummond MF, Sculpher MJ, Torrance GW, O’Brien BJ, Stoddart GL. (2005) Methods for the economic evaluation of health care programme. Third edition: Oxford: Oxford University Press.
  5. 5. Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009 Jul 21;6(7):e1000097. Epub 2009 Jul 21. pmid:19621072
  6. 6. O’Connor D, Green S, Higgins JPT (editors). Chapter 5: Defining the review question and developing criteria for including studies. In: Higgins JPT, Green S (editors), Cochrane Handbook of Systematic Reviews of Intervention. Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from www.handbook.cochrane.org
  7. 7. Guidelines for the Economic Evaluation of Health Technologies: Canada. Available at https://www.cadth.ca/media/pdf/186_EconomicGuidelines.pdf. Accessed on January 6th, 2017.
  8. 8. Husereau D, Drummond MF, Petrou SF, Carswell CF, Moher DF, Greenberg DF, et al.: Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. Value Health. 2013 Mar-Apr;16(2):231–50. pmid:23538175
  9. 9. Barnett TE, Chumbler NR, Vogel WB, Beyth RJ, Ryan P, Figueroa S: The cost-utility of a care coordination/home telehealth programme for veterans with diabetes. Journal of Telemedicine & Telecare 2007, 13(6):318–321.
  10. 10. JodarSanchez F, Ortega F, Parra C, GomezSuarez C, Bonachela P, Leal S, et al.: Cost-utility analysis of a telehealth programme for patients with severe chronic obstructive pulmonary disease treated with long-term oxygen therapy. Journal of telemedicine and telecare 2014, 20(6):307–316. pmid:25052387
  11. 11. Dixon P, Hollinghurst S, Ara R, Edwards L, Foster A, Salisbury C: Cost-effectiveness modelling of telehealth for patients with raised cardiovascular disease risk: evidence from a cohort simulation conducted alongside the Healthlines randomised controlled trial. BMJ Open 2016;6:e012355 pmid:27670521
  12. 12. Dixon P, Hollinghurst S, Edwards L, Thomas C, Gaunt D, Foster A, et al.: Cost-effectiveness of telehealth for patients with raised cardiovascular disease risk: Evidence from the Healthlines randomised controlled trial. BMJ Open 2016; 6(8): e012352. pmid:27566642
  13. 13. Cleveringa FGW, Welsing PMJ, van den Donk M, Gorter KJ, Niessen LW, Rutten GEHM, et al.: Cost-effectiveness of the diabetes care protocol, a multifaceted computerized decision support diabetes management intervention that reduces cardiovascular risk. Diabetes care 2010, 33(2):258–263. pmid:19933991
  14. 14. Boyne JJJ, Van Asselt ADI, Gorgels APM, Steuten LMG, De Weerd G, Kragten J, et al.: Cost-effectiveness analysis of telemonitoring versus usual care in patients with heart failure: The TEHAF-study. Journal of telemedicine and telecare 2013, 19(5):242–248. pmid:24163233
  15. 15. Cui Y, Doupe M, Katz A, Nyhof P, Forget EL: Economic evaluation of Manitoba Health Lines in the management of congestive heart failure. Healthcare Policy = Politiques de sante 2013, 9(2):36–50. pmid:24359716
  16. 16. Peels DA, Hoogenveen RR, Feenstra TL, Golsteijn RH, Bolman C, Mudde AN, et al.: Long-term health outcomes and cost-effectiveness of a computer-tailored physical activity intervention among people aged over fifty: modelling the results of a randomized controlled trial. BMC public health 2014, 14:1099. pmid:25342517
  17. 17. Henderson C, Knapp M, Fernandez J-L, Beecham J, Hirani SP, Beynon M, et al.: Cost-effectiveness of telecare for people with social care needs: the Whole Systems Demonstrator cluster randomised trial. Age & Ageing 2014, 43(6):794–800.
  18. 18. Dear BF, Zou JB, Ali S, Lorian CN, Johnston L, Sheehan J, et al.: Clinical and Cost-Effectiveness of Therapist-Guided Internet-Delivered Cognitive Behavior Therapy for Older Adults With Symptoms of Anxiety: A Randomized Controlled Trial. Behavior Therapy 2015, 46(2):206–217. pmid:25645169
  19. 19. Titov N, Dear BF, Ali S, Zou JB, Lorian CN, Johnston L, et al.: Clinical and Cost-Effectiveness of Therapist-Guided Internet-Delivered Cognitive Behavior Therapy for Older Adults With Symptoms of Depression: A Randomized Controlled Trial. Behavior Therapy 2015, 46(2):193–205. pmid:25645168
  20. 20. Bryan S, Mitton C, Donaldson C: Breaking the addiction to technology adoption. Health economics 2014, 23(4):379–383. pmid:24590701
  21. 21. Husereau D, Henshall C, Jivraj J: Adaptive approaches to licensing, health technology assessment, and introduction of drugs and devices. International Journal of Technology Assessment in Health Care 2014, 30(3):241–249. pmid:24921416
  22. 22. Henshall C, Schuller T: Health technology assessment, value-based decision making, and innovation. International Journal of Technology Assessment in Health Care 2013, 29(4):353–359. pmid:23845404