A major challenge of mental health care is to provide safe and effective treatment with limited resources. The main purpose of this study was to examine a value-based approach in clinical psychiatry when evaluating a process improvement initiative. This was accomplished by using the relatively new time driven activity based costing (TDABC) method within the more widely adopted cost-effectiveness analysis framework for economic evaluation of healthcare technologies. The objective was to evaluate the cost-effectiveness of allowing psychologists to perform post-treatment assessment previously performed by psychiatrists at an outpatient clinic treating depression using internet-based cognitive-behavioral therapy (ICBT).
Data was collected from 568 adult patients treated with ICBT for depression during 2013–2014. The TDABC methodology was used to estimate total healthcare costs, including development of process maps for the complete cycle of care and estimation of resource use and minute costs of staff, hospital space and materials based on their relative proportions used. Clinical outcomes were measured using the Patient Health Questionnaire depression scale (PHQ-9) before and after treatment and at 6-month follow-up. Cost-effectiveness analyses (CEA) was performed and the results presented as incremental net benefits (INB), cost-effectiveness acceptability curves (CEACs) and confidence ellipses to demonstrate uncertainty around the value of the organizational intervention.
Taking into account the complete healthcare process (from referral to follow-up assessment), treatment costs decreased from $709 (SD = $130) per patient in 2013 to $659 (SD = $134) in 2014 while treatment effectiveness was maintained; 27% had achieved full remission from depression after treatment (PHQ-9 < 5) during both 2013 and 2014 and an additional 35% and 33% had achieved partial remission in 2013 and 2014, respectively. At follow-up, 42% were in full remission after treatment during both 2013 and 2014; an additional 35% and 33% were in partial remission during 2013 and 2014, respectively. Confidence ellipses occupied the south-east (SE) and south-west (SW) quadrants of the incremental cost-effectiveness plane at both post-treatment and at follow-up, indicating that the ICBT treatment was less costly and equally effective after staff reallocation.
Treating patients to the target of full remission using psychologists instead of medical specialists for post-treatment assessment is cost-saving and consequently a more valuable use of limited resources. TDABC may be a useful tool for measuring resource costs, identifying quality improvement opportunities and evaluating the consequences of such initiatives. Combining TDABC with clinical outcome measures in CEA is potentially a useful approach in mental healthcare to estimate the value of process improvement initiatives.
Citation: El Alaoui S, Lindefors N (2016) Combining Time-Driven Activity-Based Costing with Clinical Outcome in Cost-Effectiveness Analysis to Measure Value in Treatment of Depression. PLoS ONE 11(10): e0165389. https://doi.org/10.1371/journal.pone.0165389
Editor: Yinglin Xia, University of Illinois at Chicago College of Medicine, UNITED STATES
Received: February 21, 2016; Accepted: October 11, 2016; Published: October 31, 2016
Copyright: © 2016 El Alaoui, Lindefors. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: We have uploaded a minimal dataset as Supporting Information files.
Funding: This study was supported by a research grant (ALF 20130413) from Stockholm County Council. NL received the funding.
Competing interests: The authors have declared that no competing interests exist.
The social and economic effects of the growing global burden of mental disorders—notably, depression—are monumental. The 2010 Global Burden of Disease (GBD) study identified depression as the second-leading cause of disability worldwide, concluding that depressive disorders are a public-health priority requiring cost-effective interventions. To meet the increasing demands on health care services, care delivery must be organized to maximize resource efficiency without compromising effectiveness and patient safety. An innovative solution, with the potential to bridge the gap between demand and accessibility of qualified treatment for mental health problems, is the recent development of internet-based psychotherapies, which use established evidence-based treatment programs, such as cognitive behavioral therapy (CBT), in combination with online therapist support. Such interventions are generally referred to as internet-based CBT (ICBT). [3, 4] Treatment components are similar to conventional face-to-face CBT (e.g., educating patients about depression and rationale for the CBT model of treatment, goal setting, problem solving, behavioral activation, cognitive restructuring and relapse prevention), with the difference being the novel use of internet-based communication between patient and therapist. Because ICBT is less resource demanding than conventional face-to-face interventions—and does not compromise treatment effectiveness—it could greatly increase access to evidence-based and cost-effective care, complementing traditional forms of delivery.
With an increased emphasis on mental health care in both assuring the quality of care while also managing costs, there is a corresponding need for a management system that can accurately and routinely estimate these factors. Access to detailed and reliable information on both costs accrued and treatment results achieved is critical for healthcare providers seeking to implement a value-based approach in which these critical dimensions are continuously monitored.[6, 7] Whereas measuring clinical outcomes is relatively straightforward, measuring resource use and the total costs of treating patients is more complex. Therefore, appropriate costing methods, which take the inherent complexities of health care systems into account, should be used in such estimations. Earlier studies on costing in healthcare have mainly focused on traditional hospital settings.[8–12] As such, there has been little research on cost systems in primary care and, specifically, in psychiatry. A recently developed method that may increase the accuracy of treatment cost measurement is time-driven activity-based costing (TDABC). Advantages of TDABC may include easier identification of inefficiencies and clinical improvement opportunities since a detailed inventory of each step of the care process is documented and evaluated. However, for a process improvement initiative to be of value, the impact of increased efficiency and lowered healthcare costs on the quality of care and on the clinical effectiveness of the healthcare process should always be evaluated.
To visualize the value of different treatment strategies or healthcare options, the cost-effectiveness plane (CE plane) has been widely used as a standard tool where differences in both costs and clinical outcomes between two options are illustrated . The originators of the value-based healthcare framework propose that organizations use radar charts to illustrate value, since such graphs can communicates changes in costs and outcomes on several different dimensions simultaneously . However, to be able to determine the accuracy or uncertainty surrounding the value created through process improvement initiatives, cost-saving strategies or quality improvement initiatives, we believe that the more widely adopted and recommended methods for cost-effectiveness analysis (CEA) [16, 17] coupled with the CE plane with confidence ellipses  should be used in any attempt to quantify and interpret the meaning of “value” in value based healthcare. Consequently, integrating TDABC into CEA, thus measuring value in accordance with recommended methodological guidelines, has been the focus of this study. The object of study was the evaluation of a process improvement initiative at an outpatient psychiatric unit treating anxiety disorders and depression using ICBT.
At the ICBT unit of Stockholm Health Care Services, Psychiatry Division, the effectiveness of ICBT has been well documented for a variety of mental health disorders, including depression.[19–21] The overall quality of care encompasses more than clinical outcomes; dimensions such as treatment accessibility, resource utilization and sustainability of health are equally important. Demand for treatment may vary significantly from year to year where periods with peaking demand typically lead to increased backlogs of undiagnosed patients, consequently increasing waiting times. For example, the waiting list increased by 44% as the number of yearly referrals increased from 1,026 in 2010 to 1,473 in 2013. As a consequence, the unit struggled to meet its internal quality standards of providing patients with a diagnostic assessment within three weeks and treatment availability was therefore lowered. As a process improvement initiative, a reallocation of staff was initiated in late 2013, whereby psychiatrists, who typically managed both pre- and post-treatment assessments, would focus only on clinical visits before treatment, whereas psychologists would conduct the post-treatment assessments. The objective was to increase treatment availability by reorganizing work activities performed by medical doctors allowing them to focus on more acute tasks (i.e. utilizing more of their available clinical time to complete pre-treatment diagnostic assessments) with sustained treatment effectiveness. By allowing a resource with a lower cost rate (i.e. psychologists) to perform a key activity in the healthcare process, one hypothesis was that this staff reallocation would also lead to a cost reduction.
The purpose of this study was to evaluate the usefulness of TDABC as a costing method within the cost-effectiveness analysis (CEA) framework to estimate the value of process improvement initiative for a healthcare provider perspective.
Setting and study design
This study was conducted at a public ICBT unit in Stockholm, Sweden (Stockholm Health Care Services, Psychiatry Southwest). Because of its highly standardized work processes and the high-volume nature of ICBT, an ICBT unit was considered well suited for TDABC and process management modifications. Data from 2013 and 2014 were used to compare outcomes and costs before and after staff reallocation. The research was approved by the Regional Ethical Review Board in Stockholm, Sweden (no 2011/2091-31/3).
Data was collected from patients (N = 568) who were treated for depression at the ICBT unit during 2013–2014 and completed post-treatment assessment. A majority of patients were self-referred to the clinic through an online registration and screening system, after which they were invited to a structured diagnostic interview with a clinician. To be included, patients had to (a) fulfill DSM–IV criteria of depression, (b) agree not to undergo concurrent psychological treatments, (c) have a stabilized dose of psychotropic medication for 4 weeks if on medication, (d) be able to read and write, (e) be ≥ 18 years, (f) not present with too complex co-morbid difficulties that might make ICBT an unsuitable intervention (e.g., ongoing substance abuse or a psychotic syndrome), and (g) have access to a computer or other device with an Internet connection.
To determine the cost of ICBT care in patients with depression, we followed the methodology of time-driven activity-based costing (TDABC). The following steps were applied in developing the TDABC models for each time period.
Step 1: Selecting the medical condition.
Patients diagnosed with and treated for depression was selected for this study. The care cycle included follow-up assessment at 6 months to include a measure of the sustainability of achieved treatment effectiveness.
Step 2: Defining the care delivery value chain.
An overall view of the care process was defined, including the following principal activities: diagnosis, care planning, intervention and evaluation of treatment results.
Step 3: Development of process maps.
A more detailed process map was developed for each step in the care delivery value chain, mapping processes, activities and measurements throughout the care cycle, including personnel used in each step.
Step 4: Process time estimation.
The duration for each activity was estimated through time studies (observations and interviews) for each staff category (i.e. coordinating nurse, medical secretaries, psychologists and medical doctors). Time estimates were therefore obtained for each step along the clinical pathway. The results of the time studies yielded a set of standard times for each resource. In addition, the treatment software logged the amount of time that therapists spent working online with each patient, providing access to accurate information regarding resource use during the intervention. Resource use and process times could therefore be calculated from both logged therapist time and time estimates of standardized clinical and administrative tasks, allowing estimation of the variability of total resource use.
Step 5: Estimation of resource costs.
The costs of resources used in each process were estimated. Data were collected from several sources, including administrative and medical staff, the payroll system, local business intelligence systems and the treatment software database, using 2013 as a reference year for salaries and overhead costs.
Step 6: Estimation of resource capacity and capacity cost rates.
The practical capacity of each resource was determined. The practical capacity of a resource refers to the amount of time clinically available for the resource (i.e., excluding non-treatment activities such as breaks, meetings and training), as opposed to the theoretical capacity, which is normally 40 hours per week for a full time employee. Practical capacity was estimated to be 80% of the actual number of worked hours per employee, which is typically used as a standard assumption. Calculation of practical capacity was based on attendance and wages per employee, as indicated by the payroll system. Overhead costs of hospital space, supervision, IT and management were allocated over each staff category evenly per minute. In addition, software costs for the treatment platform were allocated only to therapists. Costs for hospital space were calculated as the square meter price divided by floor space per staff category. Joint surfaces were then allocated evenly. Shared costs included management and leadership as well as shared unit administration. Hospital costs for security and safety were also included in the capacity cost rates. Capacity cost rates were operationalized as the minute cost of each resource used .
Step 7: Calculating the total cost of patient care.
The total cost of treating depression with ICBT was calculated as the sum of the total costs of resources used in each process. Costs were calculated in local currency units (SEK) and converted into international dollars.
Treatment effectiveness was measured using the nine-item Patient Health Questionnaire depression scale (PHQ-9). The PHQ-9 measures nine DSM-IV criteria of depressive disorder on a scale of “0” (not at all) to “3” (nearly every day). Remission was operationalized using established cut-off scores of the PHQ-9 where full remission was defined as a PHQ-9 score of < 5. Sustainability of treatment effect (full remission from depression) was evaluated at 6-month follow-up. Patient satisfaction was measured after treatment with the Client Satisfaction Questionnaire CSQ-8 [24, 25].
Cost-effectiveness analysis (CEA) were performed from a healthcare system perspective. The incremental cost-effectiveness ratio (ICER) was estimated at post-treatment and at follow-up. To analyze these ICERs, confidence ellipses at 50%, 75%, and 95% were developed and cost-effectiveness acceptability curves (CEACs) were constructed to represent the uncertainty around estimates  in accordance with recommended guidelines . The central limit theorem (CLT) was used in the estimation of the measures within the CEA .
The concept of incremental net benefit (INB) was used to interpret the CEAC, where the slope of the net monetary benefits (NMB) curve represents the difference in effects (ΔE) between the two study samples. An intervention was considered as cost-effective only if INB(K) > 0, where K represents the willingness to pay (WTP) for an additional unit of health gain (one more remission). Since the threshold value of the maximum WTP may be unknown, the INB of the organizational intervention compared to the 2013 processes was plotted and reported with a 95% confidence interval (CI).
The study, including its consent procedure (i.e. passive consent), was approved by the Regional Ethical Review Board in Stockholm, Sweden (no 2011/2091-31/3). The ethics committee waived the requirement for active informed consent since this research was conducted as a retrospective cohort study of patients in routine clinical practice. Instead, all participants were informed in writing of the study and given a choice of participation through an opt-out methodology. In addition, patients’ data were anonymized before access by the researchers. Since passive consent does not violate the option of providing choice and increases the likelihood of having a representative sample, this approach is considered to be an efficient procedure for registry data [28, 29].
The number of observations and PHQ-9 means for patients treated during 2013 and 2014 respectively are presented in Table 1.
During 2013, PHQ-9 scores decreased by 6.08 (95% CI: 5.38–6.77) between pre- (M = 14.69, SD = 5.00) and post-treatment assessments (M = 8.61, SD = 5.52), p < .001, corresponding to a within-group effect size of d = 1.15 (95% CI: 0.98–1.32). For 2014, PHQ-9 scores decreased by 5.55 (95% CI: 4.94–6.17) between pre- (M = 14.00, SD = 4.79) and post-treatment assessments (M = 8.45, SD = 5.70), p < .001, corresponding to a within-group effect size of d = 1.05 (95% CI: 0.90–1.19). Of patients who provided post-treatment data on the PHQ-9, 27% had achieved full remission from depression after treatment and 42% at follow-up(Table 2); there was no significant difference in the rate of remitters between 2013 and 2014 (p > .9). In addition, an additional 35% and 33% had achieved partial remission in 2013 and 2014, respectively. At follow-up, 42% were in full remission during 2013 and 2014; 35% and 33% were in partial remission during 2013 and 2014, respectively. Patient satisfaction (CSQ-8) varied non-significantly between 78% (M = 25.05, SD = 4.52) in 2013 and 79% (M = 25.43, SD = 4.58) in 2014 (p > .05).
Mean total healthcare costs of ICBT treatment for depression is reported in Table 2. Taking into account the complete healthcare process between referral to follow-up, the estimated average total cost of healthcare decreased from $709 (SD = $130) before staff reallocation in 2013 to $659 (SD = $134) in 2014. Estimated capacity cost rates (minute prices for each staff category) are presented in Table 3. The two major cost drivers were intervention costs, which contributed up to 54% of the overall costs, and assessment costs, which contributed up to 34% of the overall costs.
At post-treatment, the estimated cost-saving was $35 (SE = 12) and at follow-up $49 (SE = 16). Confidence-ellipses around the point estimate are showed in Fig 1 and in Fig 2 where each ellipse represents regions with a 50%, 75% or 95% probability of containing the true difference in cost and effect. These ellipses occupy two quadrants on the incremental CE plane: the south-east (SE) and south-west (SW) quadrants at both post-treatment and at follow-up, indicating that the ICBT treatment was less costly and equally effective after staff reallocation. The entire density within the ellipses involves cost-savings.
The cost-effectiveness acceptability curves (CEAC) for post-treatment and follow-up assessments are presented in Fig 3. These indicate the probability that the staff reallocation is cost-effective compared with before the change for a given value of the maximum willingness to pay (WTP) for a gained full remission. It may be observed that the CEACs asymptote to a value less than 1; since only 50% of the joint density involves health gains, the CEAC is decreasing as the WTP increases. This is the case when there is any evidence that the process change could be less effective (i.e. where the joint density (ΔC, ΔE) of incremental costs and incremental effects is located within the western quadrants) .
A graphical representation of the net benefit is illustrated in Fig 4. The uncertainty of the value of the intervention gets larger as the WTP for the clinical outcome increases; this is reflected in the increasing CI of the INB.
The positive NMBs (Table 4) suggest that the intervention is cost-effective, both at post-treatment and at follow-up assessment.
This study presents the use of TDABC in CEA to assess the value of process improvement initiatives in mental health care. Previous studies have established the clinical effectiveness of ICBT for depression [20, 30, 31], and the estimated effect sizes in the present study (d = 1.05–1.15) are in line with these. The main objective in this study was to observe whether treatment effectiveness can be sustained as doctors, who are in short supply and high demand, are replaced with psychologists for the activity of post-treatment assessment. Results indicated that healthcare costs decreased from $709 per treated case to $659 while treatment effects were maintained with 47% of patients achieving full remission at follow-up. Confidence ellipses occupied the south-east (SE) and south-west (SW) quadrants of the incremental cost-effectiveness plane at both post-treatment and at follow-up, indicating cost-saving where providing healthcare became less costly but equally effective after staff reallocation. The absence of an observed increase in effectiveness is reflected in the flatness of the slope of the INB line (Fig 4). Since the NMB values are positive, the process improvement initiative may be considered as cost-effective; however, the widening CI for the NMB line indicate an increasing uncertainty around the value of the intervention as the WTP threshold gets larger. In other words, the more value decision makers place on achieving better health outcomes, the less we can say about the value of the process change. However, as discussed above, the purpose of this intervention was not to increase effectiveness of the intervention, but rather to maintain the same level of effectiveness as before. In addition, it was observed that patient waiting times were reduced by approximately one week to be scheduled for a diagnostic interview and to start treatment; the median time that a patient waited for a diagnostic assessment decreased from 22 days in 2013 to 16 days in 2014 and RTT decreased from 28 days in 2013 to 23 days in 2014. Finally, patient satisfaction (from 78% to 79%) remained constant after staff reallocation.
Although these results complement previous research demonstrating that ICBT is associated with improved clinical and economic outcomes , most economic evaluations of the treatment of depression have been performed from a societal perspective, without a detailed analysis of the actual resources used throughout the care delivery cycle. Increased accuracy of cost assessments in mental health care is important in understanding the actual costs of care because such information could yield valuable insights in identifying the resources driving these costs. The purpose of using TDABC, therefore, was to achieve a more accurate method of estimating resource utilization and costs of healthcare delivery compared to more traditional costing methods such as ABC, which does not take into account the time required to perform activities in healthcare processes. By using TDABC in conjunction with established methods of CEA, we assessed its usability as a viable method for quantifying the value achieved in value-based healthcare  and taking into account the uncertainty surrounding estimated benefits of process improvement initiatives. Additional benefits of using the TDABC approach may be that healthcare providers gain a better understanding of how activities and resources are accumulated into patient level costs. Detailed information on the costs of healthcare processes may benefit strategic decision-making, allowing more insight into which process improvement initiatives are likely to lead to maintaining or improving health outcomes given available or less costly resources. For example, through this study, managers and staff at ICBT unit was provided with a detailed description of the unit’s care delivery process, its constituent parts, the resources used, their costs and the clinical outcomes. The methodology may therefore offer providers with an overall understanding of the components of their delivery system, enabling comparison and evaluation of the relative importance of each component; after all, efforts to increase efficiency in mental healthcare involves not merely cost-cutting strategies but rather improving resource allocation and utilization. Incorporating TDABC in CEA may offer further insights into which processes or resources may potentially contribute the most in terms of patient value.
We hope that this study may contribute to an understanding of—or stimulate further discussion on—how to best apply and measure the concept of “value” in mental healthcare by integrating established CEA methods into the value-based healthcare framework. The visualization of value, either on a CE plane as the widely adopted standard for health economic evaluation [16, 17] or on a radar chart, as proposed by Kaplan and colleagues , provides a fast and intuitive snapshot on the relative costs and effects between alternatives. As estimated costs and effects always involves uncertainty, we believe that presenting or reporting CIs should be included as these provide another important dimension as to the meaning of the estimates. Finally, we believe that standardized methods and guidelines of how to measure and reporting value within healthcare should be developed and agreed upon within the research community as to facilitate comparison across healthcare providers or across studies of value-based healthcare.
This study has some limitations. First, although TDABC has been extensively applied to industry [7, 9, 10], it is relatively new in healthcare, particularly mental health care. This study focused on one clinical unit, and time and cost estimates might differ significantly from other organizations within the mental healthcare field because processes differ and are not consistently standardized across healthcare providers. As a consequence, it may be difficult to generalize estimated costs of treatment or of the total healthcare process. Nevertheless, the results of this study may serve as a benchmark for other healthcare providers. In addition, one advantage of the TDABC method of measuring treatment costs—compared with traditional activity-based costing (ABC) –is that by inventorying each step of the care process, inefficient processes within the care delivery cycle may be more easily identified. Another limitation concerns the difficulties to accurately assess time and costs involved in healthcare processes. Although accurate timing of the amount of time each psychologist spent with each patient, the time estimates and costing of other administrative processes were based on averages.
To conclude, this study illustrates how a value-based framework can be implemented in a mental healthcare setting by using TDABC as a potentially more accurate costing method in conjunction with established CEA methodology. The results also showed that the quality of care could be maintained after having replaced scarce psychiatrists with more available psychologists. We encourage further studies of the use of these methods within mental health care.
S1 Dataset. Combined data set at post-treatment.
The authors gratefully acknowledge Professor Robert Kaplan and Dr. Derek Haas for reading and providing critical comments on the manuscript. We thank Mrs. Ann Edholm-Johansson, the late Mr. Stefan Lundkvist and Dr. Liselotte Jansson at Stockholm Healthcare Services, Psychiatry Division for assembling some of the study data.
- Conceptualization: SEA NL.
- Data curation: SEA.
- Formal analysis: SEA.
- Funding acquisition: NL.
- Investigation: SEA NL.
- Methodology: SEA NL.
- Project administration: SEA NL.
- Resources: SEA NL.
- Software: SEA NL.
- Supervision: SEA NL.
- Validation: SEA NL.
- Visualization: SEA NL.
- Writing – original draft: SEA.
- Writing – review & editing: SEA NL.
- 1. Ferrari AJ, Charlson FJ, Norman RE, Patten SB, Freedman G, Murray CJ, et al. Burden of depressive disorders by country, sex, age, and year: findings from the global burden of disease study 2010. PLoS medicine. 2013;10(11):e1001547. pmid:24223526; PubMed Central PMCID: PMC3818162.
- 2. Westbrook D, Kirk J. The clinical effectiveness of cognitive behaviour therapy: outcome for a large sample of adults treated in routine practice. Behaviour research and therapy. 2005;43(10):1243–61. pmid:16086979.
- 3. Hedman E, Ljotsson B, Lindefors N. Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert review of pharmacoeconomics & outcomes research. 2012;12(6):745–64. pmid:23252357.
- 4. Andersson G. Using the Internet to provide cognitive behaviour therapy. Behaviour research and therapy. 2009;47(3):175–80. ISI:000264690900001. pmid:19230862
- 5. Andersson G, Titov N. Advantages and limitations of Internet-based interventions for common mental disorders. World psychiatry: official journal of the World Psychiatric Association. 2014;13(1):4–11. WOS:000331242900002. pmid:24497236
- 6. Porter ME, Teisberg EO. Redefining health care: creating value-based competition on results. Boston, Mass.: Harvard Business School Press; 2006. xvii, 506 p. p.
- 7. Kaplan RS, Porter ME. How to Solve The Cost Crisis In Health Care. Harvard Bus Rev. 2011;89(9):46–+. WOS:000294194600027.
- 8. Kaplan RS, Witkowski M, Abbott M, Guzman AB, Higgins LD, Meara JG, et al. Using Time-Driven Activity-Based Costing to Identify Value Improvement Opportunities in Healthcare. J Healthc Manag. 2014;59(6):399–412. WOS:000345261200005. pmid:25647962
- 9. McLaughlin N, Burke MA, Setlur NP, Niedzwiecki DR, Kaplan AL, Saigal C, et al. Time-driven activity-based costing: a driver for provider engagement in costing activities and redesign initiatives. Neurosurg Focus. 2014;37(5). WOS:000344730400005. pmid:25363431
- 10. Eldenburg L, Krishan R. Management accounting and control in health care: an economic perspective. In: Chapman C, Hopwood A, Shields M, editors. Handbook of management accounting research. 2. Oxford: Elsevier; 2007. p. 859–83.
- 11. Kaplan RS, Haas DA, Krosner YC, Mukerji N. Delivering Higher Value Care Means Spending More Time with Patients Harvard Business Review; 2014 [updated December 26, 2014; cited 2015]. Available from: https://hbr.org/2014/12/delivering-higher-value-care-means-spending-more-time-with-patients.
- 12. HFM. Improving Value with TDABC 2014 [updated June 2014]. Available from: http://www.hfma.org/Content.aspx?id=22957.
- 13. Kaplan RS, Anderson SR. Time-driven activity-based costing. Harvard Bus Rev. 2004;82(11):131–+. WOS:000224746900025.
- 14. Black WC. The Ce Plane—a Graphic Representation of Cost-Effectiveness. Medical Decision Making. 1990;10(3):212–4. WOS:A1990DN45600008. pmid:2115096
- 15. Kaplan RS, Blackstone RP, Haas DA, Thaker NG, Frank SJ. Measuring and Communicating Health Care Value with Charts. Harvard Bus Rev. 2015;(10).
- 16. Philips Z, Bojke L, Sculpher M, Claxton K, Golder S. Good practice guidelines for decision-analytic modelling in health technology assessment: a review and consolidation of quality assessment. PharmacoEconomics. 2006;24(4):355–71. pmid:16605282.
- 17. NICE. Guide to the methods of technology appraisal. London: National Institute for Clinical Excellence; 2004.
- 18. Nixon RM, Wonderling D, Grieve RD. Non-Parametric Methods for Cost-Effectiveness Analysis: The Central Limit Theorem and the Bootstrap Compared. Health Econ. 2010;19(3):316–33. WOS:000275594700006. pmid:19378353
- 19. El Alaoui S, Hedman E, Kaldo V, Hesser H, Kraepelien M, Andersson E, et al. Effectiveness of Internet-Based Cognitive-Behavior Therapy for Social Anxiety Disorder in Clinical Psychiatry. Journal of consulting and clinical psychology. 2015. pmid:26009780.
- 20. Hedman E, Ljotsson B, Kaldo V, Hesser H, El Alaoui S, Kraepelien M, et al. Effectiveness of Internet-based cognitive behaviour therapy for depression in routine psychiatric care. J Affect Disord. 2014;155:49–58. pmid:24238951.
- 21. Hedman E, Ljotsson B, Ruck C, Bergstrom J, Andersson G, Kaldo V, et al. Effectiveness of Internet-based cognitive behaviour therapy for panic disorder in routine psychiatric care. Acta psychiatrica Scandinavica. 2013. pmid:23406572.
- 22. (WHO) WHO. CHOosing Interventions that are Cost Effective (WHO-CHOICE) 2015 [cited 2015]. Available from: http://www.who.int/choice/costs/ppp/en/.
- 23. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. Journal of general internal medicine. 2001;16(9):606–13. pmid:11556941; PubMed Central PMCID: PMC1495268.
- 24. Attkisson CC, Zwick R. The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Evaluation and program planning. 1982;5(3):233–7. pmid:10259963.
- 25. Nguyen TD, Attkisson CC, Stegner BL. Assessment of patient satisfaction: development and refinement of a service evaluation questionnaire. Evaluation and program planning. 1983;6(3–4):299–313. pmid:10267258.
- 26. Fenwick E, O'Brien BJ, Briggs A. Cost-effectiveness acceptability curves—facts, fallacies and frequently asked questions. Health Econ. 2004;13(5):405–15. pmid:15127421.
- 27. Glick HA, Briggs AH, Polsky D. Quantifying stochastic uncertainty and presenting results of cost-effectiveness analyses. Expert review of pharmacoeconomics & outcomes research. 2001;1(1):25–36. pmid:19807505.
- 28. Clark AM, Jamieson R, Findlay IN. Registries and informed consent. The New England journal of medicine. 2004;351(6):612–4; author reply -4. pmid:15295059.
- 29. Cassell J, Young A. Why we should not seek individual informed consent for participation in health services research. Journal of medical ethics. 2002;28(5):313–7. pmid:12356960; PubMed Central PMCID: PMC1733638.
- 30. Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World psychiatry: official journal of the World Psychiatric Association. 2014;13(3):288–95. WOS:000342851200019. pmid:25273302
- 31. Andersson G, Hedman E. Effectiveness of Guided Internet-Based Cognitive Behavior Therapy in Regular Clinical Settings. Verhaltenstherapie. 2013;23(3):140–8. WOS:000324893200003.
- 32. Kaplan RS, Bruns W. Accounting and Management: A Field Study Perspective: Harvard Business School Press; 1987.