The PACE trial compared the effectiveness of adding adaptive pacing therapy (APT), cognitive behaviour therapy (CBT), or graded exercise therapy (GET), to specialist medical care (SMC) for patients with chronic fatigue syndrome. This paper reports the relative cost-effectiveness of these treatments in terms of quality adjusted life years (QALYs) and improvements in fatigue and physical function.
Resource use was measured and costs calculated. Healthcare and societal costs (healthcare plus lost production and unpaid informal care) were combined with QALYs gained, and changes in fatigue and disability; incremental cost-effectiveness ratios (ICERs) were computed.
SMC patients had significantly lower healthcare costs than those receiving APT, CBT and GET. If society is willing to value a QALY at £30,000 there is a 62.7% likelihood that CBT is the most cost-effective therapy, a 26.8% likelihood that GET is most cost effective, 2.6% that APT is most cost-effective and 7.9% that SMC alone is most cost-effective. Compared to SMC alone, the incremental healthcare cost per QALY was £18,374 for CBT, £23,615 for GET and £55,235 for APT. From a societal perspective CBT has a 59.5% likelihood of being the most cost-effective, GET 34.8%, APT 0.2% and SMC alone 5.5%. CBT and GET dominated SMC, while APT had a cost per QALY of £127,047. ICERs using reductions in fatigue and disability as outcomes largely mirrored these findings.
Comparing the four treatments using a health care perspective, CBT had the greatest probability of being the most cost-effective followed by GET. APT had a lower probability of being the most cost-effective option than SMC alone. The relative cost-effectiveness was even greater from a societal perspective as additional cost savings due to reduced need for informal care were likely.
Citation: McCrone P, Sharpe M, Chalder T, Knapp M, Johnson AL, et al. (2012) Adaptive Pacing, Cognitive Behaviour Therapy, Graded Exercise, and Specialist Medical Care for Chronic Fatigue Syndrome: A Cost-Effectiveness Analysis. PLoS ONE 7(8): e40808. doi:10.1371/journal.pone.0040808
Editor: Pieter H. M. van Baal, Erasmus University Rotterdam, The Netherlands
Received: February 8, 2012; Accepted: June 13, 2012; Published: August 1, 2012
Copyright: © McCrone et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work supported by United Kingdom Medical Research Council, Department of Health for England, Department for Work and Pensions, and Scottish Chief Scientist Office. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: PDW has done voluntary and paid consultancy work for the United Kingdom Departments of Health and Work and Pensions and Swiss Re (a reinsurance company), and led a randomised controlled trial about graded exercise therapy funded by the Linbury Trust. TC has received royalties from Sheldon Press and Constable and Robinson. MS has done voluntary and paid consultancy work for the United Kingdom government, has done consultancy work for the insurance company Aegon and has received royalties from Oxford University Press. PM, ALJ, KAG, and MK declare that they have no conflicts of interests. This does not alter the authors' adherence to all the PLoS ONE policies on sharing data and materials.
Chronic fatigue syndrome (CFS) describes a condition of chronic disabling fatigue for which there is no other explanatory condition. It has a prevalence of 0.2–2.6% of people worldwide . CFS can disrupt employment and necessitates support from families in addition to formal healthcare , . Therapies have been developed for CFS, particularly cognitive behaviour therapy and graded exercise therapy . In the PACE trial we compared the clinical effectiveness of these two therapies with adaptive pacing therapy, when added to specialist medical care, and with specialist medical care alone . Adding cognitive behaviour therapy or graded exercise therapy to specialist medical care was found to be more effective in reducing both fatigue and disability than adding adaptive pacing therapy or specialist care alone.
The aims of this paper are to compare (i) the one-year service and societal costs associated with specialist medical care (SMC) plus cognitive behaviour therapy (CBT), SMC plus graded exercise therapy (GET), SMC plus adaptive pacing therapy (APT) and SMC alone, and (ii) the one-year cost-effectiveness of CBT, GET, APT and SMC in terms of gains in quality-adjusted life years (QALYs) and reductions in fatigue and disability.
Study Design and Participants
PACE was a parallel four-arm, multi-centre, randomised controlled trial with participants recruited from consecutive new outpatients attending six secondary care specialist CFS clinics in the UK. Participants were selected using the Oxford diagnostic criteria for CFS which required disabling fatigue to be the primary problem, in the absence of an exclusionary medical or psychiatric diagnosis . Other inclusion criteria were: aged 18 years or more, a binary score of 6 or more out of 11 on the Chalder fatigue questionnaire , and a score of 65 or less out of 100 on the Short Form-36 physical function sub-scale . Exclusionary criteria and allocation procedures have previously been described . Written consent was obtained from all participants. Ethical approval was given by the West Midlands Multi-centre Research Ethics Committee (MREC 02/7/89).
Treatment manuals are available at www.pacetrial.org. These specified at least three sessions of SMC for all participants and up to 15 individual therapy sessions for those allocated to APT, CBT and GET. SMC was provided by CFS doctors and consisted of provision of information about CFS, advice for coping, and symptomatic pharmacotherapy. APT involved management of energy expenditure and activity, under occupational therapist supervision, and aimed at helping the patients to adapt by ‘pacing’ their activity. CBT, delivered by clinical psychologists or nurse therapists, aimed to change behavioural and cognitive factors assumed to be responsible for perpetuating symptoms and disability. GET, delivered by physiotherapists, aimed to return the participant to an appropriate level of physical activity by increasing exercise in a gradual and personalised manner.
Quality-adjusted life years (QALYs) were generated from the EQ-5D health-related quality of life questionnaire at baseline, and at 12, 24 and 52 weeks after randomisation . The EQ-5D measures the following domains: mobility, self-care, usual activities, pain/discomfort, anxiety/depression. Each was coded as 1 (no problem), 2 (moderate problems) or 3 (severe problems) and UK-specific utility weights attached . The accrual of QALYs was calculated using area under the curve, assuming a linear change between each available time point. Differences in baseline utility scores were controlled for when making comparisons between treatment groups .
Interpretation of condition specific outcome measures in economic evaluations is difficult and to aid interpretability we assessed the cost per person achieving a clinically important change. Two variables were created to indicate whether patients achieved a two-point improvement on the Chalder fatigue questionnaire (CFQ) and an eight-point change on the Short Form-36 physical function sub-scale (SF36 PF) , . These changes were assumed to be clinically significant by White et al .
Service Use and Costs
In our analysis we adopted both a healthcare and a societal perspective. (The latter includes lost employment and unpaid informal care as well as health costs.) The number and duration of APT, CBT, GET and SMC treatment sessions actually delivered were recorded and time added for preparation, related correspondence, and supervision. It was assumed that the ratio of time spent on patient-related versus support activities was 1:0.3 and that capital and administrative overheads were 46% . The cost per hour of therapy was £110 for CBT and £100 for APT and GET. The cost of SMC was based on the cost per hour of consultant physician time in face-to-face contact with patients, which was £169 .
Services used during the six months before randomisation and during the 12 months after randomisation were measured with the Client Service Receipt Inventory (CSRI) . (The CSRI was used at 6 and 12 months after randomisation and the data were combined.) Services are listed in Appendix S1. Where available, unit costs were obtained from nationally recognised sources , . Specific types of medication (analgesics, antidepressants, anxiolytics, and hypnotics) were recorded and average costs assumed for each type .
Unpaid informal care from family/friends was measured by asking patients how many hours of care were provided because of fatigue. Alternative methods exist for valuing informal care, with the opportunity cost and replacement cost approaches being the most recognised. We adopted the former and valued informal care at £14.60 per hour based on national mean earnings . Days lost by patients from work, and reduced hours while at work, due to fatigue were also recorded. The human capital approach was used with the value of lost work-time to society assumed to be reflected by national mean age and gender-specific wage rates and combined with the lost employment data to generate lost production costs .
We excluded welfare benefits or payments from other sources such as private pensions and income protection schemes from the economic costs. However, we do report the receipt of these given that they are important financial outlays.
Comparisons were made between SMC alone and each of APT, CBT and GET. Healthcare, informal care and societal costs (including lost production and informal care) during the 12 months after randomisation in each treatment group were compared using regression models controlling for baseline costs and clustering for centre. Confidence intervals were generated around the cost differences using non-parametric bootstrapping .
Cost-effectiveness analyses combining data on incremental costs and outcomes can be compared for each treatment group in turn against each of the others. If costs for one treatment were lower and outcomes better than another treatment, that treatment was defined as ‘dominant’. If costs were higher (lower) and outcomes better (worse) then an incremental cost-effectiveness ratio (ICER) was calculated (i.e. extra cost divided by extra outcome). The ICER indicates the cost per QALY gained or cost per unit reduction in fatigue or disability for one treatment against another. The threshold used to assess the QALY ICERs was £30,000. ICERs constructed with the CFQ and SF-36 PF data used the differences in proportions achieving clinically important changes. The resultant ICER indicates the cost of one extra person achieving such a change as a result of using APT, CBT or GET in addition to SMC compared to SMC alone.
Interpretation of the cost-effectiveness results was made using cost-effectiveness acceptability curves . Net benefit values were computed for each study participant, defined as the value of a QALY multiplied by the number of QALYs gained minus the cost (from both healthcare and societal perspectives). We used QALY values ranging from £0 to £60,000 in increments of £5000. For each QALY value, regression models were used to determine the difference in net benefit between the four treatment arms, controlling for baseline utility and costs. Bootstrapping with 1000 resamples allowed the proportion of resamples showing APT, CBT, GET and SMC as having the highest net benefit (and to be most cost-effective) to be computed and plotted.
Sensitivity analyses were conducted around key parameters in the analyses about which assumptions had been made. Specifically we (i) estimated the cost of therapy required to reverse the findings from the initial analysis, (ii) used the minimum wage rate (£5.93 per hour) and the unit cost of a homecare worker to value informal care, (iii) used the minimum wage rate to value lost production, (iv) reduced the cost of standardised medical care by 50% to reflect the possibility of it being provided by a less senior doctor.
Role of the Funding Source
The sponsors of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. All named authors had access to the data, commented drafts, and approved the final report. Members of the writing group had responsibility for submitting the report, and PM had final responsibility for the decision to submit for publication.
641 patients were recruited, one of whom withdrew consent. Demographic and clinical characteristics of participants were similar across treatments, apart from a shorter duration of illness in SMC (Table 1). Costs and QALYs were available for 570 (89%) participants (ranging from 85% GET to 93% SMC). Those for whom both cost and QALY data were available were significantly more likely to be of Caucasian ethnicity (94.6% v 86.4%, fishers exact test p = 0.027). There were no other statistically significant differences in the background characteristics shown in Table 1. Further details of the sample have been reported previously .
Service Use and Lost Employment before Randomisation
During the 6 months before randomisation, most participants had used primary care services and over 40% other (secondary care) doctors (Table 2). Around two-thirds had used other health service professionals, between one-quarter and one-third had used complementary healthcare practitioners; most used medication. The intensity of service use (mean contacts for those using them) revealed no substantial differences between treatment groups. Lost employment was common in all treatment groups.
Service Use and Lost Employment after Randomisation
During the 12 months after randomisation, SMC participants had a higher mean number of specialist medical care contacts than those allocated to additional therapy. The number of therapy contacts did not differ between APT, CBT and GET groups. Other service use did not greatly differ between treatments during this period, although informal care hours for APT and SMC were higher than for CBT and GET. There was no clear difference between treatments in terms of lost employment.
Costs are shown in Table 3. Controlling for baseline, healthcare costs after randomisation were significantly lower for SMC than for APT (difference £840, 95% CI £637 to £1117), CBT (difference £904, 95% CI £613 to £1205) and GET (difference £829, 95% CI £534 to £1165). The differences between the APT, CBT and GET groups were small and non-significant.
Informal care costs of patients allocated to APT were significantly higher than for CBT (difference £1580, 95% CI £139 to £3132) and GET (difference £1588, 95% CI £442 to £2694). Patients allocated to SMC also had higher informal care costs than CBT (difference £1165, 95% CI £289 to £2194) and GET (difference £1173, 95% CI £740 to £1569). Lost production costs were significantly higher for APT compared to CBT (difference £1279, 95% CI £141 to £2772). Societal costs (i.e. healthcare, informal care and lost production costs) were significantly lower for patients allocated to CBT compared to APT (difference £2607, 95% CI £432 to £5585). Other differences were not statistically significant.
Welfare Benefits and Other Financial Payments
Receipt of benefits due to illness or disability increased slightly from baseline to follow-up (Table 4). Patients in the SMC group had the lowest level of receipt at baseline but the figures at follow-up were similar between groups. Relatively few patients were in receipt of income-related benefits or payments from income protection schemes and differences between groups were not substantial.
APT, CBT and GET each resulted in improvements in health-related quality of life (measured with the EQ-5D) while SMC produced little change (Table 5). CBT produced the largest QALY gain, significantly more than SMC. After controlling for baseline utility, the difference between CBT and SMC was 0.05 (95% CI 0.01 to 0.09). No other differences between treatment groups were statistically significant. The number (%) of patients achieving a clinically significant reduction in fatigue in each group was: APT 96 (64.0), CBT 113 (76.4), GET 123 (79.9), and SMC 98 (64.9). This difference was statistically significant (Fisher’s Exact test, p = 0.002). The number (%) of patients achieving a clinically significant reduction in disability in each group was: APT 75 (49.0), CBT 105 (71.0), GET 108 (70.1), and SMC 88 (57.9). This difference was also statistically significant (Fisher’s Exact test, p<0.001).
Cost-effectiveness from a Healthcare Perspective
At a threshold of £30,000 per QALY, CBT had a 62.7% likelihood of being the most cost-effective option from a healthcare perspective followed by GET at 26.8% (Figure 1). APT had a 2.6% likelihood of being most cost-effective, which was less than the figure for SMC (7.9%).
The ICERs showing the healthcare cost per QALY for CBT and GET compared to SMC were both below the threshold of £30,000 while the ICER for APT compared to SMC was substantially higher (Table 6). The healthcare costs per extra person with a clinically significant reduction in fatigue and disability are also shown. It is clear that achieving such a reduction for one person is associated with a much lower cost, compared to SMC, for CBT or GET than it is for APT. In fact, SMC dominates (i.e. has better outcomes and lower costs) APT with regard to disability.
Cost-effectiveness from a Societal Perspective
Again at a threshold of £30,000 per QALY, CBT had a 59.5% likelihood of being the most cost-effective option from a societal perspective (Figure 2). GET had a likelihood of 34.8% while APT and SMC had likelihoods of 0.2% and 5.5% respectively.
CBT and GET both dominated SMC from a societal perspective with regard to QALYs gained and reductions in fatigue and disability. Compared to SMC, APT had an incremental cost per QALY substantially higher than the £30,000 threshold while the cost per person with a clinically significant reduction in fatigue was high. SMC dominated APT with regard to disability.
The healthcare costs per QALY gained for CBT and GET compared to SMC were below the cost-effectiveness threshold of £30,000. The cost of CBT would need to increase by 45% and GET by 22% for the cost per QALY to reach £30,000. Therapy costs for APT would need to fall 35% for APT to have a cost per QALY compared to SMC of £30,000. No other sensitivity analyses (i.e. changing the value of informal care, lost employment and standardised medical care) had a large impact on cost-effectiveness.
We found that adding APT, CBT or GET to SMC resulted in significantly increased healthcare costs. For CBT and GET the healthcare costs per QALY gained were lower than the conventional £30,000 threshold used in England, indicating cost-effectiveness; for APT the cost per QALY was in excess of this threshold.
The major contributors to societal costs were lost employment and informal care costs. The cost-effectiveness of CBT was more apparent from this perspective than from the narrower healthcare perspective; CBT dominated APT, GET and SMC, while GET dominated APT and SMC. Informal care costs were substantial for each group and significantly lower after receiving CBT and GET when compared to APT and SMC. CFS affects patients in many ways and can have a major effect on family members. This study suggests that CBT and GET could ameliorate this effect. However, with the exception of a difference between CBT and APT, there were no significant differences in either lost work time or benefits between the treatments during follow up. In fact, benefits increased across all four treatments.
There are few previous studies with which to compare our results. The additional healthcare costs per QALY gained with CBT and GET compared to SMC alone in this study were £18,374 and £23,615 respectively. These costs are substantially lower than the figure of an additional €51,642 per QALY reported for CBT compared to usual care in a Dutch study . The QALY gain for CBT over usual care in that study was slightly less than we found which may account for some of this difference, given that average costs were similar. Two other studies have assessed cost-effectiveness of treatments for CFS , . However, only a minority of patients in both of these studies had CFS and neither used QALYs as an outcome measure.
This study has found that CBT and GET are cost-effective options for treating patients with CFS. However, for patients to benefit from these therapies there needs to be investment to provide the staff trained to deliver them. The findings we report suggest that such investment would be justified in terms of improved quality of life of patients and would actually be cost saving if all costs including societal costs are considered.
The study has limitations. First, we relied on self-reported information on service use and lost employment. There may be issues of accuracy with this approach but it was largely unavoidable given the need for a comprehensive perspective. Other studies have shown this to be an acceptable method –. The accuracy of our cost estimates was enhanced by the direct recording of the number and duration of therapy sessions which formed around half of the costs. Second, average medication costs were assumed for the cost periods. While greater variation would be expected, medication costs are small compared to other service costs. Third, we used the EQ-5D to generate QALY values. This is a recommended method in England, but the sensitivity of the measure in relation to changes in clinical measures in the CFS area has not yet been established. Fourth, we made assumptions regarding the value of unpaid care from family and friends and lost employment. However, sensitivity analyses revealed that the results were robust for alternative assumptions. Fifth, lost employment was valued using the human capital approach. It has been argued that this may overestimate such costs when there is high unemployment and that costs should be confined to the ‘friction period’ during which new employees can be recruited . However, what is evident is that lost work days did not show much difference between the groups and alternative approaches, while possibly reducing absolute societal costs, would not change the cost-effectiveness results. Sixth, there is similar uncertainty around the most appropriate way of valuing informal care . Alternative approaches were used in the sensitivity analyses and these did not make a substantial difference to the results. We adopted the opportunity cost approach and used average wages to reflect this value. Seventh, we analysed data only for those participants where we had data at both baseline and follow-up. This may have introduced some distortions to the results but there were few differences between patients with missing data and those on whom we had complete data. Finally, although one-year follow-up is longer than that obtained in most previous trial of treatments for CFS, we cannot be certain about the longer-term cost-effectiveness of these treatments.
At a conventional cost-per-QALY threshold considering a one year outcome CBT has the highest probability of being the most cost-effective treatment option for CFS when given as a supplement to SMC and compared to SMC alone. GET has a lower probability of being the most cost-effective option but is more likely to be so than APT or SMC alone. The probabilities that CBT GET are the most cost-effective options at a societal level are higher still, largely due to cost-savings from reducing the care required from family members.
Unit costs used in PACE study.
We would like to thank all members of the PACE trial management group. This consisted of the authors of this paper as well as (in alphabetical order): Brian J Angus (University of Oxford), Hannah Baber (Queen Mary University of London), Jessica Bavinton (Barts Healthcare NHS Trust), Mary Burgess (South London and Maudlsey NHS Foundation Trust), Julia C DeCesare (Queen Mary University of London), Lucy V Clark (Queen Mary University of London), Diane L Cox (University of Cumbria), Eleanor Feldman (John Radcliffe Hospital), Gabrielle Murphy (Royal Free Hospital NHS Trust), Maurice Murphy (Barts Healthcare NHS Trust), Hazel O’Dowd (Frenchay Hospital NHS Trust), Tim Peto (University of Oxford), Laura Potts (King’s College London), Rebecca Walwyn (King’s College London), and David Wilks (Western General Hospital).
Conceived and designed the experiments: PM MS TC MK PDW KAG ALJ. Performed the experiments: PM. Analyzed the data: PM. Wrote the paper: PM MS TC MK PDW KAG ALJ.
- 1. Prins JB, van der Meer JW, Bleijenberg G (2006) Chronic fatigue syndrome. Lancet 367: 346–355. doi: 10.1016/s0140-6736(06)68073-2
- 2. McCrone P, Darbishire D, Ridsdale L, Seed P (2003) The economic cost of chronic fatigue and chronic fatigue syndrome in UK primary care. Psychol Med 33: 253–261. doi: 10.1017/s0033291702006980
- 3. Collin SM, Crawley E, May MT, Sterne JAC, Hollingworth W (2011) The impact of CFS/ME on employment and productivity in the UK: a cross-sectional study based on the CFS/ME National Outcomes Database. BMC Health Serv Res 11. (217).
- 4. National Institute for Health and Clinical Excellence (2007) Clinical guideline CG53. Chronic fatigue syndrome/myalgic encephalomyelitis (or encephalopathy): diagnosis and management. Available: http://guidance.nice.org.uk/CG53. Accessed 2010 Nov 6.
- 5. White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, et al. (2011) Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. Lancet 377: 823–836. doi: 10.1016/s0140-6736(11)60096-2
- 6. Sharpe MC, Archard LC, Banatvala JE, Borysiewicz LK, Clare AW, et al. (1991) A report – Chronic fatigue syndrome. J Roy Soc Med 84: 118–121.
- 7. Chalder T, Berelowitz G, Hirsch S, Pawlikowska T, Wallace P, et al. (1993) Development of a fatigue scale. J Psychosom Res 37: 147–153. doi: 10.1016/0022-3999(93)90081-p
- 8. McHorney CA, Ware JE, Raczek AE (1993) The MOS 36 item short form health survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care 31: 247–263. doi: 10.1097/00005650-199303000-00006
- 9. The EuroQol Group (1990) EuroQol: a facility for the measurement of health-related quality of life. Health Policy 16: 199–208. doi: 10.1016/0168-8510(90)90421-9
- 10. Dolan P, Gudex C, Kind P, Williams A (1995) A social tariff for EuroQol: results from a UK population survey. York: University of York.
- 11. Manca A, Hawkins N, Sculpher MJ (2005) Estimating mean QALYs in trial-based cost-effectiveness analysis: the importance of controlling for baseline utility. Health Econ 14: 487–496. doi: 10.1002/hec.944
- 12. Curtis L (2010) Unit costs of health and social care. Canterbury: PSSRU.
- 13. Beecham J, Knapp M (2001) Costing psychiatric interventions. In Thornicroft G, editor. Measuring mental health needs. London: Gaskell. 200–224.
- 14. Department of Health (2011) NHS reference costs 2009–2010. Available: http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_123459. Accessed 2011 Oct 3.
- 15. NHS Information centre (2011) Prescription cost analysis, England –2010. Available: http://www.ic.nhs.uk/pubs/prescostanalysis2010. Accessed 2011 Oct 3.
- 16. Office for National Statistics (2010) Annual survey of hours and earnings (ASHE) - 2010 results. Available: http://www.statistics.gov.uk/statbase/Product.asp?vlnk=1951. Accessed 2011 Oct 3.
- 17. Barber JA, Thompson SG (1998) Analysis and interpretation of cost data in randomised controlled trials: a review of published studies. Brit Med J 317: 1195–1200. doi: 10.1136/bmj.317.7167.1195
- 18. Briggs AH (2001) A Bayesian approach to stochastic cost-effectiveness analysis. Int J Technol Assess 17: 69–82.
- 19. Severens JL, Prins JB, van der Wilt GJ, van der Meer JWM, Bleijenberg G (2004) Cost-effectiveness of cognitive behaviour therapy for patients with chronic fatigue syndrome. Q J Med 97: 153–161. doi: 10.1093/qjmed/hch029
- 20. Chisholm D, Godfrey E, Ridsdale L, Chalder T, King M, et al. (2001) Chronic fatigue in general practice: economic evaluation of counseling versus cognitive behaviour therapy. Br J Gen Pract 51: 15–18.
- 21. McCrone P, Ridsdale L, Darbishire L, Seed P (2004) Cost-effectiveness of cognitive behavioural therapy, graded exercise and usual care for patients with chronic fatigue in primary care. Psychol Med 34: 991–999. doi: 10.1017/s0033291704001928
- 22. Calsyn RJ, Allen G, Morse GA, Smith R, Templehoff B (1993) Can you trust self-report data provided by homeless mentally ill individuals? Evaluation Rev 17: 353–366. doi: 10.1177/0193841x9301700306
- 23. Goldberg RW, Seybolt DC, Lehman A (2002) Reliable self-report of health service use by individuals with serious mental illness. Psychiatr Serv 53: 879–881. doi: 10.1176/appi.ps.53.7.879
- 24. Patel A, Rendu A, Moran P, Leese M, Mann A, et al. (2004) A comparison of two methods of collecting economic data in primary care. Fam Pract 22: 323–327.
- 25. Koopmanschap MA, Ineveld BMV (1992) Towards a new approach for estimating indirect costs of disease. Soc Sci Med 9: 1005–1010. doi: 10.1016/0277-9536(92)90131-9
- 26. Koopmanschap MA, van Exel JN, van den Berg B, Brouwer WB (2008) An overview of methods and applications to value informal care in economic evaluations of healthcare. Pharmacoeconomics 26: 269–280. doi: 10.2165/00019053-200826040-00001