Reader Comments

Post a new comment on this article

Remember the two major problems with this study

Posted by matsreimer on 16 Nov 2011 at 20:44 GMT

The media reports about this trial usually omit that the study was negative in its primary end-point: self reported fatigue after three months. The trial protocol was designed this way because two of the three patients in a pilot study had early positive responses, within 12 weeks after infusion.

Four months after the last patient in the controlled trial had received treatment the study was unblinded to one of the researchers in the department. Then the decision to prolong the study’s (semi)blinded phase was made, because some patients demonstrated a late response, as had been seen in the third patient in the pilot study.

When the by then semi-blinded trial was extended, because the primary endpoint was negative, only 30 % of the patients had completed the 12-month evaluation, and the rest continued in a study that was no longer strictly double-blinded. The two main authors were unblinded somewhat later, when all patients had completed 8 months, but by then half of the patients had still not completed 12 moths.

There is always a risk that unblinding of patient allocation can cause subconscious bias in both doctor and patient. (The patients were not consciously unblinded until 12 months after treatment.)

The other serious problem with the present study is that the main finding, the much lager proportion of responders in the rituximab group, was based on definitions of major and moderate responses that were made post hoc and not defined beforehand in the protocol. (The authors mention this limitation in their discussion.)

Apart from these two major problems, which make the results rather difficult to interpret, I have a few comments of lesser importance.

The paper states that visual analogue scales were used. This was not the case, according to protocol the patients made a numerical estimation (0-6) of improvement or deterioration.

In the application to the ethics committee the researchers did not mention their patent application. (But it is reported as a competing interest in the paper.)

It has been reported by Norwegian TV that one of the patients (who received rituximab and had a positive response) was employed in the same ward as the treatments took place.

The Bergen team are now conducting two open studies of rituximab in CFS. To me it seems more important that independent researchers perform a larger truly double-blinded RCT.

References
Study protocols submitted to ethics committee (in Norwegian).
E-mail contact with the two main authors.

No competing interests declared.

RE: Remember the two major problems with this study

TomBoyles replied to matsreimer on 18 Nov 2011 at 10:18 GMT

A study result such as this challenges us to be the best possible scientists we can be.

The disease is in many ways mysterious and courts controversy. Patients advocate groups and clinicians sometimes have an uneasy relationship.

We must focus on the hard facts that this RCT showed a positive signal. We must also consider the limitations of the study (as outlined above).

From this I beleive an objective scientist would consider this a signal that B-cell depletion may be effective in CFS but would be worried by the use of post-hoc definitions, of partial unblinding and particularly by the authors' pecunary interest in the results.

In the best interests of patients with this highly debilitating condition this study needs to be repeated by an independent group with no financial interest in a fully blinded fashion with pre-determined definitions and end points. Open label studies by interested parties are very likely to show positive results but are unlikely to move the science forward.

No competing interests declared.

RE: Remember the two major problems with this study

Fluge replied to matsreimer on 21 Nov 2011 at 16:06 GMT

In his comment to our article, dr. Mats Reimer wants to highlight the limitations of our study. These have also been described by us in the manuscript.

Our study is small with only 30 patients included. We agree, as stated previously, that there is a need for a large, placebo-controlled, randomised study, with inclusion of patients fulfilling strict diagnostic criteria for CFS/ME. We hope to be able to participate in a multi-centre, randomised study in Norway and an initiative has been taken to initiate this.

As dr. Reimer states, the primary endpoint of our study was negative, as there was no difference in self-reported Fatigue-score at three months after intervention between the Rituximab and Placebo groups. As described in the manuscript, the protocol was designed when we had experience from only two pilot patients, both with an ”early” response pattern. Therefore the primary endpoint was chosen at three months, with secondary end-points after 2, 4, 6 and 12 months. After observing the third pilot patient, with a ”late” but major response on all symptoms, starting to occur from 5 ½ months after Rituximab infusion (Fluge and Mella, BMC Neurology 2009), an amendment was approved by the ethical committee to include secondary endpoints after 8 and 10 months.

According to the protocol, the code for intervention could be inspected after all the patients had completed 4 months follow-up (end of October 2009). Due to the experience of a ”late” response in the third pilot patient, we were unsure if our assumption of main response at 3 months was correct. Therefore, professor Olav Dahl, who did not have any patient contact and was responsible for safety in the study, inspected the code for intervention and the electronic patient files.

At the time Olav Dahl inspected the code (end of October 2009), all patients had completed 4 months follow-up, 19 had completed 6 months follow-up, 12 had completed 9 months follow-up, and 9 patients had completed 12 months follow-up. Upon his decision, the blinding of the study was extended until all patients had completed 8 months follow-up (end of February 2010). The decision was based on his interpretation that there was an association between group allocation and clinical improvement for those observed more than 6 months after intervention, while there seemed to be no difference for those observed only 4 months.

The main authors ØF and OM, who assessed the patients during follow-up, were blinded for the intervention code until end of February 2010 (all patients had then completed 8 months follow-up, 19 patients had completed 10 months follow-up, and 13 patients had completed 12 months follow-up). Importantly, the patients were blinded for group allocation until all patients had completed 12 months follow-up (minimum 12 months, up to 20 months). We must remind of the fact that the results of this study were (except for the physicians’ assessments) based on the written symptom registration by the patients. We do not believe that the blinding procedures described have influenced the main conclusions of the study.

As described in the article, and also in our response to comments in Plos One, the criteria for overall response, with respect to extent and duration of improvement, was not prespecified in the protocol. At the time the protocol was designed, we did not have the available information to predefine this, having observed only two patients.
Cut-off for classification of response versus non-response is therefore exploratory and post-hoc. However, the reported cut-off for overall response was in accordance with the interpretations of clinical meaningful improvements, both by the patients and by us.
The General Linear Model (GLM) for repeated measures of Fatigue-score is directly based on the patients’ self-reported symptoms during follow-up. This analysis showed a significant interaction between time and intervention group, demonstrating that the Fatigue score during the follow-up was different in the Rituximab and Placebo groups. This analysis is not exploratory or post-hoc and therefore a main result of the study.

Dr. Reimer is worried about that the fact that one of our patients had worked in the Department of Oncology and Medical Physics. She had been out of work due to CFS/ME. Haukeland University Hospital is the biggest employer in Hordaland County (where all patients were recruited from), with about 12.000 persons employed. We strongly reject the assumption that the inclusion of a patient employed at the hospital, and who had 50% chance to be allocated to either of the groups, should influence the results of the study.

Dr. Reimer refers to the initial application to the Regional Ethical Committee from January 2008 and that the possible patent applications were not mentioned in this document. In December 2007, we informed Bergen Teknologioverføring (BTO Consulting, a company owned by our employer Haukeland University Hospital, the University of Bergen and the Institute of Marine Research in Bergen) about the possibility to investigate if a patent application could be relevant on the issue of B-lymphocyte depletion in CFS/ME. In January 2008 this was considered very preliminary and was not mentioned in the initial application to the Regional Ethical Committee. This patent application was promoted by BTO and not by the authors, but ØF and OM are named as ”inventors” in the patent applications. In later articles, lectures or presentations, we have stated this as a possible conflict of interest.
We refute the assumption by both dr. Reimer and dr. Boyles, that the patent application promoted and owned by BTO, is an obstacle for our participation in future clinical studies. There is reason to believe that studies elsewhere in the world also will be initiated to investigate the concept of B-lymphocyte depletion in CFS/ME and that these data will broaden the knowledge of immune manipulation in this debilitating disease. The end result of this will be a confirmation or rejection of the conclusions of our small study, in line with how medical knowledge is accumulated in all other areas of medicine.
We are now performing an open-label phase II study to evaluate Rituximab induction followed by maintenance treatment. We do not agree that such a study has limited scientific value, as suggested by dr. Boyles. One important rationale for this study is the need for further knowledge on the patterns of responses and relapses, to be able to optimise the design of a larger randomised, double-blind and placebo-controlled study. All our studies are performed with collection of biological material from the patients and hopefully this will help us to better understand the pathophysiology of the disease in the future.

Øystein Fluge and Olav Mella

Competing interests declared: Haukeland University Hospital has patents and pending patent applications on the issue of B-cell depletion therapy for chronic fatigue syndrome. Family members of WO2009083602 A1 are pending, as well as granted US 12/348024. The two authors ØF and OM are named as inventors in these applications.