Reporting of Systematic Reviews: Better Software Required

This is an important paper and editorial [1,2]. Systematic reviews should be much more widespread, and not only for randomized clinical trials of clinical treatments. A paper on an elegant piece of experimental data or on epidemiological observations would be made all the more interesting if the first table were a high-quality assessment of previous studies. In fact, I would suggest that performing a systematic review should be part of a research protocol for any subject, even before the study is initiated. However, this paper confirms my suspicion that the rising popularity of “systematic reviews” has not been followed by adherence to methodological rigor. 
 
With this background, I would like to point out one weakness that may explain part of the current quality deficit in some systematic reviews. There is a substantial lack of software that can assist in an important part of a systematic review: tracking literature searches and early phase screening. From browsing of the literature and communications with various Norwegian and Danish Cochrane collaborators (including the RevMan developers), there seems to be a limited number of tools for this use. Oftentimes, it is suggested that commercial reference management software be used, such as the popular EndNote. These types of software were not designed with systematic reviews in mind. At later stages of a review, Cochrane's RevMan is useful, but not early on. 
 
As far as I have been able to ascertain, there are only two tools presently available. The first is EPPI-Reviewer (http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=184), which is non-profit, but does not seem to be open source or available for local deployment. The second is TrialStat's SRS software (http://www.trialstat.com), which is commercial and has a substantial price tag. 
 
I would therefore encourage researchers and institutions to contribute to the development of open-source tools for assisting in systematic reviews. I am currently writing such a simple tool, based on the open-source JabRef package (http://sourceforge.net/projects/jabref) and would welcome feedback on perceived needs and other similar projects.


Jan Brogger
This is an important paper and editorial [1,2]. Systematic reviews should be much more widespread, and not only for randomized clinical trials of clinical treatments. A paper on an elegant piece of experimental data or on epidemiological observations would be made all the more interesting if the fi rst table were a high-quality assessment of previous studies. In fact, I would suggest that performing a systematic review should be part of a research protocol for any subject, even before the study is initiated. However, this paper confi rms my suspicion that the rising popularity of "systematic reviews" has not been followed by adherence to methodological rigor.
With this background, I would like to point out one weakness that may explain part of the current quality defi cit in some systematic reviews. There is a substantial lack of software that can assist in an important part of a systematic review: tracking literature searches and early phase screening. From browsing of the literature and communications with various Norwegian and Danish Cochrane collaborators (including the RevMan developers), there seems to be a limited number of tools for this use. Oftentimes, it is suggested that commercial reference management software be used, such as the popular EndNote. These types of software were not designed with systematic reviews in mind. At later stages of a review, Cochrane's RevMan is useful, but not early on.
As far as I have been able to ascertain, there are only two tools presently available. The fi rst is EPPI-Reviewer (http:⁄⁄eppi.ioe.ac.uk/cms/Default.aspx?tabid=184), which is non-profi t, but does not seem to be open source or available for local deployment. The second is TrialStat's SRS software (http://www.trialstat.com), which is commercial and has a substantial price tag.
I would therefore encourage researchers and institutions to contribute to the development of open-source tools for assisting in systematic reviews. I am currently writing such a simple tool, based on the open-source JabRef package (http://sourceforge.net/projects/jabref) and would welcome feedback on perceived needs and other similar projects.

University of Bergen Bergen, Norway
We have read with great interest the work of Sandbulte et al. recently published in your journal [1]. In this article, the authors provide evidence for the existence of cross-immunity between the neuraminidase of H5N1 viruses and that of endemic human H1N1 viruses. Age may be an important determining factor in the development of cross-immunity: younger people, having a shorter history of H1N1 exposure, may be disproportionately susceptible to H5N1 infection.
We would like to highlight the infl uence of the geographicdependent epidemiological behaviour of infl uenza in the development of cross-immunity. While Europe, the United States, and northern Asia experience regular outbreaks of infl uenza each year, ("seasonal infl uenza"), infl uenza in tropical regions such as southern China, Vietnam, and Indonesia tends to be year-round ("non-seasonal" infl uenza). In consequence, the probability of exposure to infl uenza A in these regions persists throughout the entire year. Repetitive contacts with infl uenza wild viruses could promote the development of cross-immunity against different viral strains. Even more, it could represent a fortuitous mechanism for developed natural protection by the close and persistent exposure of the immune system to infl uenza wild viruses in regions known for being an important source of emergent viruses, like southern China.
Results from Sandbulte et al. show that antibodies play a dominant role in cross-protection. The authors underscore the possible benefi t of seasonal infl uenza vaccination for human populations faced with the threat of pandemic H5N1 infl uenza. This idea deserves careful analysis. The main group at risk for severe complications of seasonal fl u are people older than 65. In Western countries, this population is recommended to receive annual vaccinations. Generally speaking, elder vaccination rates in tropical countries are far lower than those in Western countries. Even with the low annual vaccination rate in elders, H5N1 infection is observed mostly in young people. The existence of sub-clinical or asymptomatic infections in elderly people cannot be ruled out, but the reason why there are no described clinical cases of H5N1 in people older than 40 years is currently unknown. An age-dependent differential distribution of avian-type receptors in the upper respiratory tract could be a possible explanation. On the other hand, Tumpey et al. [2] demonstrated that mucosal (but not parenteral) challenges with inactivated or live H3N2 virus protect against H5N1 infection in mice. These results could have a relevant consequence: does contact with circulating infl uenza A via the respiratory tract confer a higher degree of crossprotection than parenteral exposure to vaccines?
In conclusion, the non-seasonal epidemiological behaviour of infl uenza in tropical countries could dramatically infl uence the development of naturally induced cross-immunity against different infl uenza strains and diminish the risk of severe disease from new emergent strains in elderly people living in these countries. The apparent lack of H5N1 cases in the elderly may be the result of continued exposure to circulating non-seasonal infl uenza A via mucosal epithelium in the respiratory tract. Vaccination via the mucosal route could be a more effi cient way to provide cross-protection against future pandemic strains than vaccination via the parenteral route. In this hypothetical scenario, Western countries would be underprotected. Competing Interests: The authors have declared that no competing interests exist.

Lethal Injection: Let's Be Honest about the Death Penalty
Lawrence Bonchek I don't favor the death penalty, and I don't participate in executions, but I recognize that honorable people can disagree about the subject, and I don't consider physicians who wish to do so-to assure that death comes quickly and without unusual pain-to be behaving unethically. They could be seen as serving the interests of both the condemned and society. The conundrum about lethal injection persists only because long-standing American Medical Association guidelines prohibit physicians from carrying out actual executions, or even pronouncing death, though they may certify it-a distinction without a difference. The lack of physician involvement has resulted in execution protocols based on outmoded pharmacologic methods, carried out by inconsistently qualifi ed technicians, with results that are sometimes ineffective and therefore are understandably controversial.
Any qualifi ed anesthesiologist could propose more reliable techniques. It is unreasonable to assert that a condemned person cannot be executed painlessly, when tens of thousands of people are anesthetized every single day for surgery with modern fast-acting anesthetic drugs (propofol, in particular) that are far more suitable than the outmoded execution drug thiopental. Induction of surgical anesthesia does occasionally cause slight injection pain, so how then can it be "cruel and unusual" to use the same drug and method for the initial step in executions? Next, potassium cannot fail to stop the heart instantly and insensibly if given in substantial amounts.
The debate about the death penalty should be conducted about its morality, not about its methods, because the latter is merely a surrogate for serious debate. Opponents of the death penalty (like me) should recognize that it is unwise to criticize methods alone, because improved methods vitiate those arguments [1].

Funding:
The author received no specifi c funding for this article.
Competing Interests: The author has declared that no competing interests exist.

Lethal Injection: Other Views
The

PLoS Medicine Editors
The recently published research article on lethal injection [1] and our editorial commentary [2] both produced a number of short letters via our electronic reader response system. There were fi ve letters commenting on the research article. All were written by strong supporters of the death penalty who took the view that, as the victims of those who had been convicted of murder had suffered, the perpetrators should themselves experience pain as well as execution. We posted two of these letters on our Web site, but felt the other three were written in terms that made them unsuitable for inclusion.
Our editorial attracted 11 letters. Three supported the views we had expressed, one commented on a small factual error, and seven were hostile, again focusing on the desirability of making murderers suffer. We felt that one of the supportive letters (above) was of particular interest and we have therefore chosen to highlight it by means of formal publication in the Correspondence section of the journal. Of the seven hostile letters, we considered that four were suitable for posting on the Web site.
We do not intend, within our Correspondence section, to publish further letters commenting on the research article or the editorial. However, as with all the articles we publish, reader responses for the Web site may be submitted at any time. After a very brief screening for suitability, reader responses appear on the site within a day or two of submission.

The PLoS Medicine Editors
The Public Library of Science San Francisco, California, United States of America The authors deserve thanks for laying out decent principles of communication [1]. But serviceable consent language is insuffi cient to address all issues of protection. That was the point of recent workshops held by the National Institutes of Health to develop a genome-wide association studies program [2].
Risks associated with personal identifi cation may be incurred if information is subject to code breaking. Legal means are available to compel identifi cation, including across national boundaries. Privacy protections under the Health Insurance Portability and Accountability Act (HIPAA) are subject to exceptions, including for law enforcement, downstream data users, or for other reasons, and are not available internationally. Even with authorization, the complexities associated with a repository may frustrate attempts to achieve meaningful comprehension. Use of data for purposes other than pharmaceutical product development or biomedical interventions would be an abuse resulting perhaps in travel restrictions or discrimination.
For these reasons, safeguards should be added, including: • Amendments to prevent non-medical health access to personal identifi cation information; • Restrictions on recruitment of populations especially vulnerable to disclosure risks, such as prisoners or immigrants; • Prohibitions on disclosure to or use by employers or third-party payors to deny medical coverage, assign differential premium risks, restrict access to therapies, or unfairly discriminate in employment.
Another risk from creation of a genomics repository is the potential for unjust stigmatization (see for example [3]). A workable program would state that the data are appropriate only for limited public health purposes involving product development or professionally derived biomedical intervention, and are insupportable for other use or by political or non-medical entities.
A researcher publishing results based on the genomic data should state affi rmatively a boilerplate recognition of the abuse potential for stigmatization. This mechanism could prevent others from the wayward misappropriation of data for purposes other than those intended by professionals. The boilerplate could read: "Conclusions derived from the genotypic or phenotypic characterization of individuals, groups, or families in this [publication] are meaningful or supportable only for the purpose of biomedical intervention or treatment and are unethical, insupportable, or inappropriate for use in other purposes. Use of the data to support any result of stigmatization, discrimination, or adverse social harm would constitute a misuse or abuse of the data." To increase the connection of benefi ts to participants, individuals should be given personal opportunities to receive news reports if they wish and learn of particular clinical trials directed at their characteristics. If the data are to be used in the development of pharmaceutical products, users should also be directed to plan and explain early on how targeted populations may have reasonable access to treatment or therapy if the product is successfully brought to market. These suggestions are consistent with the program outlined by Senator Barack Obama in the Genomics and Personalized Medicine Act of 2006 and Senator Olympia Snowe in the Genetic Information Nondiscrimination Act of 2007 [4,5].
Improved consent: Yes, but linked to and inseparable from strong protections and added benefi ts for participants.
Robert Reinhard (rreinhard@mofo.com) San Francisco Vaccine Trials Unit San Francisco, California, United States of America recent observational studies have p-values of 0.01-0.05 [3,4]; most "positive" trials and meta-analyses also have modest pvalues.
3. I provided equations for calculating the credibility of research fi ndings with or without bias. Even without any bias, PPV probably remains below 0.50 for most non-randomized, non-large-scale circumstances. Large trials and meta-analyses represent a minority of the literature. 4. Figure 1 shows that bias can indeed make a difference. The proposed modeling has an additional useful feature: As type I and II errors decrease, PPV(max) = 1 − [u/(R + u)], meaning that to allow a research fi nding to become more than 50% credible, we must fi rst reduce bias at least below the pre-study odds of truth (u less than R). Numerous studies demonstrate the strong presence of bias across research designs: indicative reference lists appear in [5][6][7]. We should understand bias and minimize it, not ignore it.
5. "Hot fi elds": Table 3 and Figure 2 present "the probability that at least one study, among several done on the same question, claims a statistically signifi cant research fi nding." They are not erroneous. Fields with many furtive competing teams may espouse signifi cance-chasing behaviors, selectively highlighting "positive" results. Conversely, having many teams with transparent availability of all results and integration of data across teams leads to genuine progress. We need replication, not just discovery [5].
6. The claim by two leading Bayesian methodologists that a Bayesian approach is somewhat circular and questionable contradicts Greenland's own writings: "One misconception (of many) about Bayesian analyses is that prior distributions introduce assumptions that are more questionable than assumptions made by frequentist methods" [8].
7. Empirical data on the refutation rates for various research designs agree with the estimates obtained in the proposed modeling [9], not with estimates ignoring bias. Additional empirical research on these fronts would be very useful.
Scientifi c investigation is the noblest pursuit. I think we can improve the respect of the public for researchers by showing how diffi cult success is. Confi dence in the research enterprise is probably undermined primarily when we claim that discoveries are more certain than they really are, and then the public, scientists, and patients suffer the painful refutations. Much has been written about how open access to biomedical journals is vital for researchers in developing countries [1], but so much more needs to be done.
Our experience in Peru with the Health InterNetwork Access to Research Initiative (HINARI), an initiative managed by the World Health Organization that helps promote access to scientifi c information by providing free (or low cost) online access to major science journals, is not as accessible as hoped for and, in fact, is getting worse. When HINARI launched in 2003, it provided access to more than 2,300 major journals in biomedical and related social sciences [2].
In April 2007, we conducted a review of the fi rst 150 science journals available through HINARI with the highest impact factors on the Science Citation Index [3]. We excluded open-access journals and journals that make online access free to low-income countries (e.g., The New England Journal of Medicine, British Medical Journal Publishing Group). We could not access any of the top fi ve journals from major publishers such as Nature and Elsevier-Science Direct. In other words, from the Nature Publishing Group we had no access to Nature Reviews Cancer, Nature Reviews Immunology, Nature Reviews Molecular Cell Biology, Nature, or Nature Medicine, and from Elsevier ScienceDirect we had no access to Cell, Cancer Cell, Current Opinion in Cell Biology, Immunity, or Molecular Cell. In addition, we could not access any of the fi rst-level journals from Blackwell, Oxford Press University, Lippincott Williams and Wilkins, or Wiley and Sons. In 2003, all these journals were available.
Our fi ndings support comments received from users over the last 8-10 months at the main library at Universidad Peruana Cayetano Heredia (Oscar Gayoso, personal communication). Students and faculty could not get access to biomedical journals from Nature, Elsevier-Science Direct, Blackwell, Oxford Press University, Springer Science, Lippincott Williams and Wilkins, or Wiley and Sons through HINARI. The collections of journals from the abovementioned publishers together represent approximately 57% (2,118 of 3,741) of journals that were supposed to be accessible through HINARI, while the remaining 43% accessible were largely composed of open-access journals or journals that make online access free to low-income countries.
Moreover, we have found a signifi cant decrease in the number of users accessing HINARI at our institution. For example, the number of HINARI users has decreased from 12,144 in April 2005 to 5,655 in April 2007, which may refl ect the loss of impact of the HINARI initiative at our institution. In contrast, the number of users accessing other databases such as ProQuest and EBSCO has increased over the last few months.
Our fi ndings suggest that we not only have access to a reduced number of biomedical journals on HINARI, but we also have no access to the biomedical journals that have the highest impact factors. The HINARI Web site states that it is still incorporating new journal collections. However, we are afraid that any addition that will not provide access to major publishers (such as the Nature Publishing Group, Elsevier ScienceDirect, or Lippincott Williams and Wilkins) could lack real impact according to HINARI's goals.
Since 2003, Peruvian medical students and health professionals have substantially benefi ted from access to high-quality scientifi c information through HINARI. Few medical students and very few researchers in the developing world can pay the usual fee of US$20-US$45 to download one article. Not even some private universities in Peru can afford the minimum journal subscription rates, even though these subscriptions would help the universities to become less isolated from global medical research. Having to pay US$1,000 per year to HINARI has left many public universities in the provinces of Peru without access because they cannot afford it. Even for the Peruvian institutions that are currently paying US$1,000 per year to HINARI, what is the real benefi t of their HINARI subscription now?
We fear that the loss of access to many key journals that are published by the major companies could be a major setback to the education of medical students in Peru and perhaps around the world. Furthermore, it could make biomedical research in developing countries like Peru, a key element in fi ghting poverty, even scarcer.
In conclusion, students and researchers in developing countries such as Peru, working at the frontlines of global health problems, need to access more biomedical journals in order to practice evidence-based health care and conduct high-quality research. The recent loss of access to many key biomedical journals in Peru could be a step backwards. We hope the situation described in this letter might help lend support to the proposal of Godlee et al., who suggested that the World Health Organization and its partners should take the lead in establishing an international collaborative group along the lines of the Global Fund to fi ght AIDS, Tuberculosis and Malaria to achieve the goal of "Universal access to essential health-care information by 2015" or "Health information for all" [4].