PLOS policy is clear on this. The authors must share the data. It will allow their stated claims to be reasonably assessed for accuracy. This is not about discrediting a paper, it is about scientific process. Unless there is something wrong with the paper, unless the data does not support the authors claims, then there should be no anxiety over sharing the data. I can't understand the heavy resistance.

The authors benefited from the reputation of PLOS by publishing in their journal, now they want to show PLOS up by completely ignoring their agreement? I wait to see what PLOS think about that. From my point of view I wonder how I can trust this paper if no one is allowed to see the data it is based on. I am sure PLOS will not allow this to go unchallenged as to do so would leave people wondering how many other PLOS papers are suspect.

I also don't understand why QMUL in their reply to James Coyne have made reference to a FOI request that was nothing to do with him. It also seems very selective to highlight that particular FOI request when another more recent one has been upheld by the ICO in favour of the requester (currently under appeal by QMUL).

"Harrassement and distress of staff"...

What about the patients? Millions of us patients suffer from PACE trial publications including this one.

Plos One needs to take the side of good science, by following their rules, access to data by whoever is asking said data.

In this case, the institution is refusing access to the data and I concur with James Coyne that the paper should be retracted until data is provided to those requesting it.

A patients right to accurate information about treatments should be treated as more important that an academics sensitivities. Part of that decision should be about understanding the cost benefits especially if a patient is going private.Comments on this paper refer to analyses defined in the statistical analysis plan that were not reported. Hence there is work for an academic to do to look at the data further and ensure reporting according to the criteria set out by those who designed the trial.

But perhaps the key comments here are on a thread from TKindlon which points out a charge of £176 per session is made yet the authors do their calculations on a cost of £110. So the treatments do not offer value for money for clinical commissioners or patients but offer considerable profit for the Maudsley hospital which I believe is part of KCL who refused access to data.

This really is very straightforward. The authors agreed to PLOS terms and conditions when they published their article in the first place. To refuse to now comply with these terms and conditions is a breach of that contract and PLOS must retract the article until this situation is resolved or a permanent retraction is put in place. It has to be said that the consistent refusal of the PACE authors to release any data relating to a trial that has had a major impact on the medical care and health of some extremely ill patients, is beyond concerning at this stage.

There are crucial facts and context missing from QMUL's (and now KCL's in part) narrative of anti-science harassment against the PACE trial. They appear to conflate all significant criticism with harassment without any regard for the validity of the comments made.

In addition to James Coyne PhD, other senior academics and research scientists have also expressed significant concerns over how the trial was conducted, analysed, or reported. Their assessments are outlined in a series of articles written by David Tuller PhD (lecturer in journalism and public health at UC Berkeley) see http://www.virology.ws/me... . Six of them wrote an open letter to the editor of the Lancet, outlined some main problems and called for a fully independent re-analysis of the results. Similarly, over 11,000 individuals signed a petition calling for the retraction of questionable claims (made in relation to the 'normal range' for fatigue and physical function that overlapped with trial entry criteria for severe disabling fatigue), and called for the release of de-identified individual-level trial data so other researchers can re-analyse the results, see http://my.meaction.net/pe... . As Tuller, Coyne, and others have noted elsewhere, accusations of campaigns of harassment appear to arise whenever legitimate questions are asked about the trial.

As the person who submitted the FOIA request in ICO decision notice FS50558352, I wish to comment. It had nothing to do with James Coyne. It was my attempt to clarify confusion about the timing and nature of changes to the PACE trial recovery criteria, which had reached the level of parliamentary debate in the UK House of Lords in 2013, but has not been conclusively resolved. The revised recovery criteria is asserted to be pre-specified but there is good evidence that it is post-hoc and unapproved. I provided them background about how the confusion arose and then asked specific questions to conclusively resolve the matter.

QMUL presented to the ICO a detailed narrative of harassment and argued that any disclosure under the FOIA is unnecessary as they have procedures for review and dissemination. The ICO accepted QMUL's arguments about feeling harassed, deferred to their authority and judgement, focused on subjective interpretations of tone and circumstances, but did not consider the justifications for the request or the evidence that the procedures for review and dissemination were inadequate in this case. The evidence justifying the request in the first place was deemed outside the scope of the investigation simply because it involved details about the PACE trial. S.14(1) is supposed to account for the justifications for a request in order to judge whether the level of alleged disruption or annoyance caused by the request is justified. With all due respect to the ICO in general (and not to diminish my appreciation for their unequivocal support for my other request), it is difficult to successfully argue that a request or any burden it places on a public authority is justified when the evidence which justifies it is precluded out of the investigation.

Therefore in my opinion, the consideration of evidence was imbalanced, and the ICO staff involved with that particular case were mislead by a one-sided presentation of the evidence. QMUL's threshold for examples of harassment apparently includes moderated BMJ Rapid Responses and published letters to the editor etc, without any regard for the validity of the points made in the correspondence. The BMJ encourages patient involvement and does not publish responses it deems inappropriate. It appears that QMUL view all significant criticism of the PACE trial to be a form of harassment simply because they dislike or disagree with it.

It is not harassment to express legitimate concerns about research e.g. major deviations from a published trial protocol, ask sincere questions, disagree with the interpretation of research(ers), highlight factual errors, or be somewhat frank when it is called for. These are all legitimate activities and scrutiny is part of science. It is unclear how disclosing details about methodology could discredit any trial if it was conducted and reported properly. Ben Goldacre's (co-founder of AllTrials) compare-trials.org project expects details about the timing and nature of changes to protocols to be routinely disclosed.

QMUL have misguided assumptions about my beliefs and motives. I maintain that they have no convincing evidence that I intended to harass them instead of seek information to resolve an ongoing confusion and controversy in the patient community. My FOIA request was not perfect, but it is all online so people can judge for themselves whether it was unreasonable given the relevant circumstances of a post-hoc analysis possibly being described as pre-specified : https://www.whatdotheykno... .

Why have the authors responded as if you asked for data via the FOIA? I understood you invoked the PLOS datasharing requirement, which is a condition of publication. These are two entirely separate channels, and there should be no confusion between them.

If the PACE authors are concerned that their trial is being discredited, they need look no further than their own bizarre and misleading analyses that have already brought the trial into disrepute among patients and those few researchers who have been paying attention.

All of the protocol-specified main outcome analyses and the criteria for recovery were abandoned once the trial was underway. The new analyses included a threshold for recovery of physical function that is so low that it is below the level of disability required for trial entry and is close to the mean for Class II congestive heart failure patients.

Patients are relying on PLOS One to be the first scientific institution to stand up for good scientific practice in the context of the PACE trial. The principle of data-sharing is sound, and the study authors signed up to it when they submitted their paper to PLOS One.

To treat Professor Coyne's request as a Freedom of Information request, let alone one that that is "vexatious", is disgraceful.

PLOS One is being tested by the PACE authors. Please, PLOS One, stand by your policies and don't fail patients. Patients don't risk their health in clinical trials so that study authors can misrepresent the results and prevent independent researchers from investigating them.

We are severely concerned about the scientific behaviour of the researchers working at the PACE study. Why do they not present their raw data? If PACE is a real scientific research, with nothing to hide, they should welcome this request, the way all scientists should do.
We are becoming very concerned about the trustworthiness of the outcomes of this study. This, at the same time, will have an effect on the international view on the trustworthiness of the institutions the researchers work for.

Lou Corsius, Msc
The Netherlands

Solid data and good science does not crumble when exposed to polemics and people with various motives. Succeeding to discredit a medical trial, in a proper and scientific way of course, is exactly how useful information on the topic would be obtained.

It is impossible that prominent researchers and scientist are not aware of this, so in this refusal only motives that are suspicious are theirs.

In KCL's recent denial of Prof. James Coyne's request for data from the PACE trial (1), one of the decisions that they considered 'in particular' as justification was John Mitchell v Information Commissioner EA/2013/0019 (Mitchell) (2), which dealt with a request by myself for meeting minutes of the PACE Trial Steering Committee and Trial Management Group. In their decision the Tribunal stated that-

"9. Queen Mary has relied on s.36(2) FOIA which provides:-

information to which this section applies is exempt information if, in the reasonable opinion of a qualified person, disclosure of the information-
(b) would, or would be likely to inhibit-
(ii) the free and frank exchange of views for the purposes of deliberation...

11. The Commissioner in considering this concluded that it was reasonable for a qualified person to decide that disclosure of the minutes would inhibit free and frank provision of advice and free exchange of views. He therefore decided that the opinion was reasonably arrived at and he concluded that the exemption in section 36 was engaged."

The s.36(2) FOIA exemption applied by QMUL dealt with discussions of the PACE Trial Steering Committee and Trial Management Groups, the disclosure of which the ICO felt "would inhibit free and frank provision of advice and free exchange of views". It is unclear how this decision would have any bearing on KCL's denial of Prof. Coyne's inquiry since the KCL researchers published their findings in a publically accessible research journal, the entire point of which being to reach as broad an audience as possible. This is about as far as one can get from private discussions consisting of "the free and frank exchange of views for the purposes of deliberation", since the whole point of publishing one's findings is to have them subjected to public discourse, debate, criticisms, etc., a process otherwise known as 'the scientific method'. Therefore KCL's reference regarding "the need to protect academic freedom and protect academic staff who put forward ‘new ideas and controversial or unpopular opinions’" is highly questionable since having one's methodologies, calculations, and interpretations of data be analyzed and re-analyzed by any interested party who chooses to is just as much a part of, and essential to, the scientific process as the initial publication itself-

"The great advantage of the scientific method is that it is unprejudiced: one does not have to believe a given researcher, one can redo the experiment and determine whether his/her results are true or false. The conclusions will hold irrespective of the state of mind, or the religious persuasion, or the state of consciousness of the investigator and/or the subject of the investigation. Faith, defined as belief that does not rest on logical proof or material evidence, does not determine whether a scientific theory is adopted or discarded.

A theory is accepted not based on the prestige or convincing powers of the proponent, but on the results obtained through observations and/or experiments which anyone can reproduce: the results obtained using the scientific method are repeatable. In fact, most experiments and observations are repeated many times (certain experiments are not repeated independently but are repeated as parts of other experiments). If the original claims are not verified the origin of such discrepancies is hunted down and exhaustively studied." (3)

KCL then argues that an "active campaign to discredit the project has caused distress to the university’s researchers who hold legitimate concerns that they will be subject to public criticism and reputational damage", however it is hard to imagine a situation by which a researcher having their findings be re-analyzed could result in 'public criticism and reputational damage' unless the original findings were found to be flawed in some way. In fact, this argument would be just as applicable to the researcher who would be re-analyzing the data, as they would also be putting their reputation on the line should they come to a different conclusion that the original authors. If the original findings were found to be valid then there would be no reasonable expectation of 'reputational damage and criticism', and if the original findings were found to not be valid then concerns over 'reputational damage and criticism' should be no reason whatsoever to hold off conducting the re-analysis.

In fact, such a situation would be the very antithesis of the scientific method, not to mention the idea of 'academic freedoms' previously cited by KCL. 'Academic freedom' does not in any way, shape or form pertain to some sort of ill-defined and all-encompassing 'freedom' from having one's work re-analyzed or scrutinized and it is patently bizarre for KCL to suggest so, as noted by the popular blog 'Retraction Watch', which asks its readers whether "the absurd reasons (cited by KCL in their refusal) make your blood boil as much as ours". (4) Also, contrary to KCL's assertion of 'active campaigns to discredit the project' leading to 'public criticism and reputational damage', it is in fact KCL's withholding of the data that is the very source of much of the criticism directed at it, as evidenced by the outpouring of social media postings condemning KCL's denial-

- Brian Nosek, Executive Director of the Center for Open Science and Professor at the University of Virginia- "King's College data sharing refusal and rationale are antithetical to science" (5)

- Chris Hartgerink, PhD student in Methodology and Statistics at Tilburg University- "[King's College] invoke academic freedom to not share data = Outrageous. I thought King’s College was a serious academic institution." (6)

- Dr. Andy Buckley, Royal Society University Research Fellow- "KCL with a pisspoor refusal to release controversial data for replication" (7)

- Dr Caroline Dodds Pennock, Lecturer in International History, University of Sheffield- "From external POV, KCL refusal of [Prof. Coyne's] FOI request on grounds of researcher 'distress' seems ludicrous" (8)

- Richard Smith, former chief executive of the UnitedHealth Europe and the BMJ Group, former editor of the British Medical Journal 1991-2004- "King's College is surely making a mistake in holding back data from a controversial trial." (9)

Regardless, even if re-analysis of the data was certain to result in 'reputational damage and criticism' of the investigators, the entire subject is irrelevant as there is no 'reputational damage and/or criticism' exemption to the FOIA. The ICO's Guidance on Dealing with vexatious requests (section 14) clearly states that "The key question to ask yourself is whether the request is likely to cause a disproportionate or unjustifiable level of distress, disruption or irritation". (10) As noted above, since the idea of having one's findings subjected to outside scrutiny is both an inherent part of and integral to the scientific process itself, for KCL to categorize such scrutiny and/or re-analysis of research data as causing "disproportionate or unjustifiable level of distress, disruption or irritation" is incompatible with both the letter and spirit of the exemption. As noted by the Editors of PLoS One, the journal in which the paper in question was published- "Some raise the concern that, having collected data, they want to be the ones to analyze it and benefit from it. In our view, this sentiment applies to the period before publication. But after publication (in particular, after publication in an Open Access journal) the data should be available for re-use by others. This is not just our view: many institutions and funding agencies (e.g. NIH) now make data sharing a requirement. We understand that some authors will not want to share data, just as some choose not to make their articles available Open Access, but trust that most authors publish their work precisely in order to allow others to benefit from it." (11)

While KCL acknowledges that the PACE trial has led to controversy, it is becoming increasingly dissatisfactory for KCL to claim that criticisms of the PACE authors are the result of "active campaigns to discredit the project" as KCL is either unaware of or neglects to mention the substantial criticisms the PACE trial continues to incur from senior academics and research scientists (12,13,14,15), up to and including an open letter to the Lancet requesting independent re-analysis of the individual-level PACE trial data- "We therefore urge The Lancet to seek an independent re-analysis of the individual-level PACE trial data, with appropriate sensitivity analyses, from highly respected reviewers with extensive expertise in statistics and study design. The reviewers should be from outside the U.K. and outside the domains of psychiatry and psychological medicine. They should also be completely independent of, and have no conflicts of interests involving, the PACE investigators and the funders of the trial." (16) Other such criticisms include the following-

- Dr. Bruce Levin, Columbia University: “To let participants know that interventions have been selected by a government committee ‘based on the best available evidence’ strikes me as the height of clinical trial amateurism.”

- Dr. Ronald Davis, Stanford University: “I’m shocked that the Lancet published it…The PACE study has so many flaws and there are so many questions you’d want to ask about it that I don’t understand how it got through any kind of peer review.”

- Dr. Arthur Reingold, University of California, Berkeley: “Under the circumstances, an independent review of the trial conducted by experts not involved in the design or conduct of the study would seem to be very much in order.”

- Dr. Jonathan Edwards, University College London: “It’s a mass of un-interpretability to me…All the issues with the trial are extremely worrying, making interpretation of the clinical significance of the findings more or less impossible.”

- Dr. Leonard Jason, DePaul University: “The PACE authors should have reduced the kind of blatant methodological lapses that can impugn the credibility of the research, such as having overlapping recovery and entry/disability criteria.”

While KCL also cites in its denial "the view of the Information Commissioner in decision FS50558352 that the request in that case was ‘more focussed on attacking and attempting to discredit the trial than in obtaining useful information on the topic’", a decision which was strongly rebutted by the requester, (17) KCL is again either unaware of or neglects to mention several other similar recent FOI requests for PACE trial data which were initially denied only to be overturned at a later date either by internal review or by subsequent ICO rulings. (18,19)


1. https://dl.dropboxusercon...
2. http://www.informationtri...
3. http://physics.ucr.edu/~w...
4. http://retractionwatch.co...
5. https://twitter.com/Brian...
6. https://twitter.com/chart...^tfw
7. https://twitter.com/agbuc...^tfw
8. https://twitter.com/carol...^tfw
9. https://twitter.com/Richa...^tfw
10. https://ico.org.uk/media/...
11. http://blogs.plos.org/eve...
12. http://www.virology.ws/20...
13. http://www.virology.ws/20...
14. http://www.virology.ws/20...
15. http://news.sciencemag.or...
16. http://www.virology.ws/20...
17. http://www.ncbi.nlm.nih.g...
18. https://www.whatdotheykno...
19. https://ico.org.uk/media/...

I do not suffer from ME. Nor does any of my family or close personal friends (with the exception of one lady whom I visit in my capacity as a clergyman). I have no scientific or medical training or competencies. I do hope that those who consider themselves harassed or traduced by the request for detailed data understand just how this looks to the complete outsider with nothing more than a human concern for those with ME. Whatever the "truth" of the quality of analysis of the findings of the relevant trial a refusal to share the data suggests having something to hide. Whatever the quality of the research and researchers and whatever their motivation, a defensive (and aggressive) response to questioning/criticism/opposition again suggests a vulnerability which goes beyond human pride. ME seems to me to be a debilitating and extremely serious condition which needs to be investigated in a spirit of open scientific inquiry - and this situation appears to me, a lay observer, to deny that basic element of the scientific approach, namely an open and open-minded ethos. I'm sorry, but neither PLOS ONE nor the authors of the report appear to me to come out of this with any credit. I can contribute nothing else to this debate due to my admitted ignorance but I felt it was of sufficient importance to warrant taking the time to open an account and to make this small addition to the topic.