Reader Comments

Post a new comment on this article

Exclusion of CHERRIES guideline

Posted by plosmedicine on 31 Mar 2009 at 00:28 GMT

Author: Gunther Eysenbach
Position: Senior Scientist,
Institution: Centre for Global eHealth Innovation
Submitted Date: July 12, 2008
Published Date: August 11, 2008
This comment was originally posted as a “Reader Response” on the publication date indicated above. All Reader Responses are now available as comments.

Being the author of a guideline on conducting Web-based surveys (CHERRIES - Checklist for Reporting Results of Internet E-Surveys) [1] - I was surprised to find that our CHERRIES guideline was not included, and that I was never contacted by these researchers. In fact, it does not even show up in their list of excluded guidelines. This is despite the fact that our CHERRIES guideline is listed on the EQUATOR website since their launch, so these authors must have been aware of the CHERRIES guideline. The CHERRIES guideline has been developed and adopted by the Journal of Medical Internet Research (, which I edit, to help improving the quality of reports from Web- or email surveys. The guideline was developed after consultations with editorial board members and authors, and was published as an editorial (rather than buried in the instructions for authors or circulated only as internal guideline) in a hope that other authors and editors – not only for our journal - will adopt these guidelines widely, regardless for what journal they are writing. In fact, the guidelines had an impact that go well beyond adoption at one specific journal (see e.g. [2]).
As such I am stunned that it was missed (or excluded without being listed under “excluded”?) by the EQUATOR team.

Simera et al’s inclusion/exclusion criteria for guidelines in this EQUATOR survey seems very arbitrary, highly subjective, and non-systematic.
While Simera et al aimed to include " initially [..] all available guidance for reporting of scientific studies in health research.”, which they subsequently narrowed down to include “only very broad, generic guidelines", I fail to see why guidelines for reporting results of acupuncture RCTs or tumour marker prognostic studies can be considered "broader" and "more generic" than guidelines for reporting the results of web surveys developed and adopted by a medical journal. Simera et al included guidelines on conduct and reporting of survey research [3], yet excluded our guideline on web-based survey research, which is an increasingly common research method with various important reporting requirements which are often missed by authors or editors (in fact, Simera et al’s study was also an e-survey, though not web-based but email-based).
I am still not sure of their omission of CHERRIES was an error or a deliberate omission. If it is the latter, then this reveals a failure to acknowledge the central role that journals and journal editors have role to play in the development, dissemination, adoption, and perhaps even funding of reporting guidelines – after all, reporting research is their core business, and journal editors – at least when they take their job seriously - spend a considerable amount of time trying to improve the reporting of research. Attempts by journal editors to identify gaps in reporting quality and to widely disseminate checklists or guidelines that address these gaps should be embraced, not ignored.
I also believe most of the adoption and implementation problems addressed in box 2 can be overcome when groups developing guidelines collaborate closely with journal editors from the outset, or if guidelines originate as part of journal operations.
If it was a deliberate omission, then one also has to ask why "guidelines developed by journals or journal editors" are not listed as one of their exclusion criteria. Not reporting this (pretty significant) exclusion criterion seem a major omission to me.
Thirdly, if this guideline was excluded deliberately, then I am confused as to why it is not listed in their list of excluded guidelines? How many other guidelines were screened and excluded but not listed as excluded?
Fourthly, if the objective of this research was to "survey the authors of these guidelines to gather details about their development methodology, dissemination and implementation strategies", why is an entire category of guidelines - those developed and advocated by journal editors - (quietly) excluded from this survey? If the idea of this survey is to find out how guidelines were developed, than it should be as inclusive as possible, and not a priori exclude an entire class of guidelines or guideline authors (those developed by journals, journal editors, and their respective researcher communities).

I admire the intention of this paper and the EQUATOR project, but this particular survey strikes me as a very subjective and perhaps biased approach to a topic that deserves a more careful and systematic approach. I will be careful in reading too much into the deliberate or erroneous omission of a editor/journal developed guideline (which raises the question of how many other such guidelines were ignored), but if the EQUATOR network wants to succeed to “act as an ‘umbrella’ organisation, bringing together developers of reporting guidelines, medical journal editors and peer reviewers”, than it appears unwise to exclude guidelines that have been developed by journal editors.

Gunther Eysenbach MD, MPH
Editor & Publisher, Journal of Medical Internet Research

Senior Scientist, Centre for Global eHealth Innovation

Associate Professor,
Department of Health Policy, Management and Evaluation, University of

Mailing address:
Centre for Global eHealth Innovation
R. Fraser Elliott Building, 4th Floor, room # 4S435,
190 Elizabeth Street
Toronto, ON M5G 2C4
phone (+1) 416-340-4800 Ext. 6427
fax (+1) 416-340-3595


1. Eysenbach G Improving the Quality of Web Surveys: The Checklist for Reporting Results of Internet E-Surveys (CHERRIES)
J Med Internet Res 2004;6(3):e34

2. Shah A, Jacobs DO, Martins H, Harker M, Menezes A, McCready M, Pietrobon R. DADOS-Survey: an open-source application for CHERRIES-compliant Web surveys. BMC Med Inform Decis Mak. 2006; 6: 34.

3. Kelley K, Clark B, Brown V, Sitzia J (2003) Good practice in the conduct and reporting of survey research. Int J Qual Health Care 15: 261-266.

No competing interests declared.