Paul Chinnock is Managing Editor of the Cochrane Injuries Group (London, United Kingdom). Nandi Siegfried is a South African Nuffield Medical Fellow at the University of Oxford (Oxford, United Kingdom). Mike Clarke is Director of the United Kingdom Cochrane Centre (Oxford, United Kingdom). The authors are involved in the work of the Cochrane Collaboration, but the views expressed in this article are their own and are not necessarily those of the Cochrane Collaboration.
The authors' employment depends upon the value and importance given to systematic reviews.
Systematic reviews have yet to achieve their potential as a resource for practitioners in developing countries, argue Chinnock et al.
Although there is still some resistance to the evidence-based medicine movement, evidence-based health care has now become widely accepted and adopted. Systematic reviews of the effectiveness of health care interventions are the engine room of evidence-based health care; much has been written about how these reviews should be conducted and what they can achieve [
But are the systematic reviews that have so far been published relevant and of practical use to those who provide health care in “the majority world” (i.e., in developing countries? In our view, the relevance of systematic reviews to frontline health care workers in developing countries has so far been limited, for a number of reasons.
Most of the reviews produced to date address health conditions that are priorities in the developed world [
(Photo: World Health Organization/P. Virot)
Health care professionals in developing countries sometimes wonder whether their reliance on older, cheaper, “lower-tech” approaches has made their practice quite distinct from that of their colleagues in richer regions [
Systematic reviews are based largely on research that has been done in rich countries. One of the reasons for this is the relative lack of research in developing countries. However, even when research has been conducted in these countries, it might not be published [
Once studies have been found, they are assessed for quality by the reviewers. Only when the quality meets the criteria specified in the review protocol (in most cases, this specifies randomised controlled trials only) are they included in the analysis. The difficulties of conducting randomized controlled trials in resource-poor situations result in the exclusion of many developing country studies. Some have suggested that the “quality threshold” should be lowered, so that more studies from developing countries can be included in systematic reviews. This question is contentious, and indeed divides the authors of this essay, but it needs to be recognised and debated openly.
Practitioners in low-income countries have questioned the “transferability” of evidence derived from studies conducted in richer nations [
Features of the typical health care experience of a patient living in the developing world, as compared with features of the typical health care experience of a patient in a clinical trial in a developed country, are shown in
Features of the typical health care experience of a patient living in the less developed world include
late presentation self-medication of “prescription” drugs or traditional treatments poor facilities may delay diagnosis referral (if needed) not easily arranged if a child, may be malnourished if a woman, may be anaemic will experience problems because of shortages of trained staff …and because of poor infection control …and because of a lack of follow-up care patient may be unable (e.g., because of lack of funds) to fully adhere to treatment.
Features of the typical health care experience of a patient in a clinical trial in a developed country include
none of the above
There are also important differences in the way in which care is delivered in developing and developed countries. In developing countries, treatments that would be delivered by doctors elsewhere are often delivered by medical assistants or clinical officers. This may or may not have an impact on the effectiveness of the treatment. Similarly, legislation can be considered a health care intervention for the prevention of road traffic injury, but the “delivery” of such legislation (i.e., its enforcement) is often harder to achieve in developing countries for a multitude of reasons.
As a result of such differences, the most effective treatment in a randomised controlled trial may not be the most effective treatment when provided in the developing world. Some treatments will retain much of their effectiveness in a resource-poor context; others will not.
One recently updated Cochrane review on the primary repair of penetrating colon injuries is a case in point [
Rather than implying that a review's conclusions are globally applicable, perhaps this is one of those circumstances where it would be more appropriate if reviewers concluded with statements such as, “There is evidence for the effectiveness of this intervention in the countries and setting where the included studies were conducted, and in places that are similar in terms of the resources available.”
It is, of course, vital that more research of quality and relevance is conducted in developing countries, but the writers of systematic reviewers also have much to do. We need to find ways to make a good product better, and we must do more to make sure that people in the majority world are able to access the reviews that are published. In order for progress to be made, the following questions require more attention than they have received up to now.
How can we involve more people from developing countries in the writing and peer reviewing of systematic reviews? For example, how can we continue to build on progress made on international activity within the Cochrane Collaboration [
Data used in this table are from [
How can we get more reviews written on (a) health problems that are priorities, and (b) interventions that are affordable and feasible in the majority world?
Should reviews focus on specific contexts in relation to the location of the condition and the delivery of the intervention?
How can we encourage reviewers to look at conditions/interventions globally, and not just as they affect the United States and Western Europe?
How can we make it easier to find and review data from research done in developing countries?
Should reviewers be encouraged to consider whether heterogeneity between study results might be due to differences in underlying resources?
Should conclusions address whether any recommendations apply everywhere, or just in settings similar to those in which the included studies were done? Or is this beyond the recommendations of a review?
Is this best done by circulating the reviews themselves, or are reviews merely a stage in the production of more accessible evidence-based health information materials? For example, the World Health Organization's Reproductive Health Library, available on CD-ROM, includes selected Cochrane reviews but also summaries and commentaries that have been specially prepared to provide a developing world perspective. The BMJ's Clinical Evidence produces other summaries of the evidence (for example, often integrating the findings of Cochrane Reviews into answers to clinical questions), and aims to prepare these in user-friendly formats and languages. Are more initiatives like these needed?
Research is needed on the impact of systematic reviews on practice in the developing world. We need to assess: What proportion of reviews are relevant to health care in low-resource settings? Are evidence-based sources used to set policy in different countries? How widely are the Cochrane Library and/or Cochrane reviews used by health care workers, and what are the barriers to use? How widely are these resources used by other people involved in decisions about health care, including patients, their carers, and policy makers? Has the use of Cochrane evidence influenced practice? What do these users and potential users think would make reviews more useful?
When so-called developing countries first gained freedom from their colonial oppressors, Ernst Schumacher pointed out that there was a need, not for the “best” technology, but for “appropriate” technology [
With thanks to Fred Bukachi, Luis Gabriel Cuervo, and Neil Pakenham-Walsh for their comments on previous drafts of this essay.