Reader Comments

Post a new comment on this article

If CBT works and is Cost-Effective Why Does the Data Not Show This?

Posted by Kgeraghty on 30 Jan 2013 at 14:55 GMT

Taking a simplistic and pragmatic look at this paper, one might expect that if CBT and or GET were very successful treatments for CFS we would see a large drop in the numbers of CFS patients using primary care services post treatment (eg GP visits post treatment). However, the data does not show this at all, hence the contention that CBT is cost effective does not marry with the data.

Is Pacing (APT) a Therapy in the First Instance?
Within this paper the control is Pacing Therapy – this is something patients do on their own, at home; they fall ill with ME/CFS and pace themselves according to their daily illness/symptom fluctuations. There is no such thing as Formal Pacing Treatment, thus when comparing CBT with APT, the authors are comparing a non-treatment with a treatment, hence making comparison of results impossible. Further still, the authors of PACE had great control over prescribing treatments and allocating therapy, eg the training of CBT/GET therapists, the running of centres and so on. This generates considerable bias within PACE that has spilled over into this latest cost-effectiveness analysis.

Can Pacing really be less cost effective compared with CBT/GET?
CBT is generally given in clinics and centres. The cost to set up and maintain these centres runs into the millions, if not hundreds of millions yearly. If we start at baseline that all groups, the CBT, the SMC, and the GET groups, have a fixed societal cost of £11,500, we see that all we do by offering a formalised treatment called APT (Pacing Therapy) is to more than double this figure to over £23,000. Yet, pacing is something people do at home, on their own, this is why it’s calling pacing; it is not a prescribed validated treatment based on any sort of model, it is patients self-limiting activity. If we accept this fact there is no need for a formalised APT programme, hence costs should remain at baseline £11,807. If we then view that GPs/doctors send their CFS patients off for CBT treatment, which we know to be only moderately effective from the PACE findings, we can then judge CBT being almost double the cost of APT (£11,807 for home-pacing v £20,288 for clinic CBT) [SD UK Pounds 09-2010] for only minimal benefits to the patient in terms of improvements in symptom reduction of physical functioning. A more pragmatic analysis of APT v CBT or GET is that pacing is roughly half the cost. It is only formalised pacing support that costs slightly more. The lead author (an economist) does not seem to address this point.

Finally, if CBT and GET were truly effective treatments for CFS we would expect post-treatment primary care and hospital use to fall significantly across the CBT and GET groups post 12 months, yet this is not supported by the data. We see small falls across most groups, including the APT and SMC groups. A pragmatic interpretation of the data would be that CBT and GET do not greatly reduce GP visits and hospital visits, hence it could not to an effective treatment for CFS (this has been noted by other comments within this forum and references have been given to alternative CBT studies that show it is often ineffective; and these alternative studies are supported by patient surveys which show a similar trend, that CBT and GET are largely ineffective).

This cost-effectiveness analysis of this paper is largely based on multiple complex statistical suppositions about what things ‘might’ cost, and then applying health economics, such as the use of QUALYs (which are generally used for studies of drug interventions cost v benefit). Unfortunately, the complexities of CFS/ME mean costs and cost effectiveness are much more complex issues than would be found in a study of a new drug to prolong life in cancer treatment. Issues of patient inclusion criteria, sampling, bias, and heterogeneity are far more complex in CFS studies. By comparing a non-treatment (APT) to a treatment (CBT) this paper forwards a finding that is largely based on the constructs of the PACE authors, rather than on clinical realities, and as such, retains much of the same bias inherent in PACE.

From this paper’s findings, APT at home is almost 50% less costly compared with either CBT or GET.

It would be interesting to see if specialist ME/CFS support couselling (SMEC) (from a non-psychiatry derived model, that includes understanding of the biological abnormalities eg Maes and Twisk's Model) might work compared v GET or GET.

Competing interests declared: Director of Small ME/CFS Research Group, ME/CFS Researcher.

RE: If CBT works and is Cost-Effective Why Does the Data Not Show This?

spjupmc replied to Kgeraghty on 30 Jan 2013 at 20:28 GMT

I would like to thank Dr Geraghty for the time he has taken to comment on our paper. Unfortunately his arguments do not stand up to scrutiny. In response:

1. Dr Geraghty has a general concern that the PACE trial did not evaluate home-based PACING. As a non-clinician it is difficult for me to say what form of PACING is most appropriate but we could extend his argument and say we could also have examined other forms of CBT and GET delivery. (CBT for example is frequently delivered for other patients via computers or the internet.) We always been very clear about the interventions we have compared. It would be unscientific to make assumptions about the relative cost-effectiveness of interventions we have not examined.

2. A simple reading of the paper would reveal that the costs at baseline are reported over a 6-month period while the follow-up costs are for 12 months. Therefore GET total costs at baseline of £11,523 and at follow-up of £20,935 do not represent a doubling at all - the period which the latter figure relates to is twice as long.

3.To assume that cost-effectiveness is the same as cost saving is incorrect. This may happen, but interventions with increased costs are frequently judged as cost-effective if the outcomes are sufficiently improved (as was the case for CBT and GET).

4. An economic evaluation has to consider all relevant costs. In his comments Dr Geraghty focusses on primary care and hospital costs which are decreased for all groups (remembering that the follow-up is 12 months). However, family costs are substantially decreased for CBT and GET. It is vital that the impact on carers is not ignored as they are key providers of care. (Although, it is worth emphasising that even from a narrow healthcare perspective CBT and GET are clearly more cost-effective than APT or SMC alone.)

5. QALYs are absolutely not confined to evaluations of drugs. They are the standard measure of effectiveness which NICE uses in its assessments of all health technologies unless there are specific reasons not to. There is a debate to be had about the use of QALYs per se but that is certainly not specific to this study.

Competing interests declared: Lead author

RE: RE: If CBT works and is Cost-Effective Why Does the Data Not Show This?

Kgeraghty replied to spjupmc on 31 Jan 2013 at 04:49 GMT

Dear Prof McCrone thank you for your response.

1. The point I was making seems to have been missed. PACING is something patients do on their own. It does not require a formalised plan with a support worker to show patients how to pace. It is only constructed this way in the PACE Trial by the PACE authors who were essentially seeking to compare CBT and GET against SMC and the common approach taken by patients (which is to Pace). However, in the PACE Trial Pacing becomes a treatment therapy for comparison, (this is achieved via constructing APT as a formal treatment option with associated costs) yet in reality there is no real need to formalise pacing. GPs for example, could offer advice to patients on 'how to pace' this would then make APT part of the SMC Care Group and the majority of the costs involved in providing formalised APT would be saved. Then if one was to compare un-formalised APT v formalised CBT, the findings about cost-effectiveness would be very different.

In this regard we find that the PACE Trial is constructed in such a way that the type of results found are predetermined by the type of study constructed (this is the essential bias). This cost-effectiveness study is based on those results, yet clinical reality does not marry up with either the PACE study findings or the findings of this study. It may well be that formalised APT is slightly less cost effective than CBT when one adds in hypothetical costs to society, but just as one could add in these facts, one could also argue that there is no need for formalised APT hence comparing APT v CBT shows that patient-based pacing (self pacing) is much more cost effective than formalised CBT.

If CBT were substantially effective as a treatment for CFS we might see results like dramatic reductions in primary care visits - yet we do not, we might see dramatic reductions in benefit claims, yet we do not (and this data is not included as far as I can see). We might also see dramatic increases in return to work/employment rates - again we do not see this.

What we have in my humble opinion, is a lot of speculation about what might be saved if we compared one construct compared against another 'researcher formulated' construct - and even within this comparison we have the issues of sampling bias. In PACE we see patients with mental health problems being included 77%. We find a bias in the selection of patients, drawn mainly from the researchers clinics. We know that bed-ridden and severely unwell CFS patients were excluded. Hence, when we talk about the findings of this paper or indeed the PACE Trial, we need to be very careful to state that what we are talking about is mildly fatigued patients with high levels of associated mental health problems, rather than the entire CFS population. We have no idea if CBT. GET, SMC, or APT are useful for the group of patients not seen by PACE, those too sick to participate in PACE.

Returning to clinical reality. If CBT and GET were very effective treatments we would find this confirmed by patient surveys, yet again this is not the case, and to the contrary patient are saying CBT helps some patients with coping, yet does not return them to normal health. Worse still, patients often report CBT and GET being harmful, we have no reports of harms within PACE or this study.

There are two realities in conflict here. The world view generated by PACE and then this study with it's analysis of cost effectiveness based on the PACE data v the world view generated by patients and clinical realities. I would direct you to read 'Why Most Published Research Findings Are False' within Plus Med.

Ioannidis JPA (2005) Why Most Published Research Findings Are False. PLoS Med 2(8): e124. doi:10.1371/journal.pmed.0020124

No competing interests declared.

RE: RE: If CBT works and is Cost-Effective Why Does the Data Not Show This?

Kgeraghty replied to spjupmc on 01 Feb 2013 at 23:04 GMT

From point 4 above:

Prof. McCrone places great importance on the fact this paper has included projected societal costs of carers taking time off work to care for sick CFS patients. My basic intrepretation of this analysis is that in the APT Group and the SMC Group require more care time than the CBT and GET Groups. Prof. McCrone uses an opportunity cost of a hourly rate of pay lost for every hour of care given. This is the research construct devised by this paper.

In contrast, the clinical reality is more complex: the PACE Trial on which this paper is based, examines mainly mild-to-moderately unwell CFS patients (by definition severely unwell patients too unwell to attend sesssions and clinics were excluded). For many CFS sufferers in this former mild category a significant percentage will not have a carer, either paid or unpaid. Some will be 20-40 year old people living alone or with friends. Others will live with family. In many instances family members who do not work, (eg stay at home mothers), or family members who are retired, will often offer care support. In this regard, Prof. McCrone's costings, of the societal costs (the costs of care) are inaccurate and misleading.

In order to know costs, we would need to know exactly how many of the 640 Pace Cohort actually had carers; we would need to know who those carers are and whether they were working and then decided to give up work or reduce hours in order to care. As far as I know, we do not have this data; instead we have assumptions about care given and this is turned into a societal cost; and then this is judged against perceived lower societal costs in the CBT/GET Groups.

In the real world, some primary care givers may give up work, however families and friends often juggle work and care giving, and families often find a way to offer care without reducing family income. In addition, a good number of CFS patients will self care, perhaps even live alone.

As I have mentioned above, there is a lot of supposition within this paper, yet detailed review of the assumptions made to conduct the cost-effectiveness evaluations do not mirror reality.

The findings of paper are widely reported in the media, with the NHS Website reporting CBT and GET cost-effective treatments for CFS; yet this may be an inaccurate assumption if we apply a different set of variables/assumptions, like APT mostly being self-applied (= reduciton in costs); care mostly given by home family members not working (societal costs much reduced) and so on. In this alternative scenario we might have a headline like, CBT/GET more costly, less cost-effective.

Competing interests declared: As above