Comment on King and Bertino, “Asymmetries of Poverty: Why Global Burden of Disease Valuations Underestimate the Burden of Neglected Tropical Diseases”

King and Bertino (hereafter KB) are right that global burden of disease (GBD) estimates involve many arguable assumptions and cannot take account of every way in which it is bad to be sick or hurt. However, their criticism of the concept as applied to neglected tropical diseases is full of mistakes and misunderstandings, the most important of which this letter corrects. References are to page numbers in the 24-page version available at http://www.plosntds.org/a....

In Box 3 on p. 6, KB state three of what they call the "Hidden Assumptions of the DALY Approach". In the first place, none of the assumptions that go into the GBD estimates are hidden; all are in plain view in the extensive documentation which KB cite but appear never to have read with any care. More to the point, assumptions 2 and 3 are utterly and completely wrong, so wrong that it is hard to imagine how KB came up with them.

Assumption 2 is that "There is a linear association between resource investment in a control program and the improvement of disease burden." The estimation of GBD includes no assumptions whatsoever about control programs, most certainly not that their cost is a linear function of their scale or coverage. There are perhaps two possible explanations for this colossal error. One is that since all people are regarded equally, apart from differences in age and a small difference in life expectancies between men and women, the total reduction in disease burden from a control program applied to a population is the sum of the individual reductions in burden of all the affected members of that population. That is, averting the deaths of 200 five-year-olds causes exactly twice the reduction in GBD as would be achieved by averting the deaths of 100 children of that age. That relation is correct, but it says nothing about the cost—the resource investment—of reaching 200 children versus only 100 of them. The other possible explanation is simply that KB do not understand the difference between GBD estimates and estimates of the cost-effectiveness of disease control programs. The cost-effectiveness of such programs is the relation between their cost and the reduction in disease burden that they achieve. It is often the case that there are too few data to trace a cost curve and conclude that marginal cost is constant and equal to average cost. But there is no presumption that costs are constant; in fact, much of the discussion in chapter 15 of Disease Control Priorities in Developing Countries, 2nd edn, which KB cite (their second and 28th references, unnumbered) but seem also not to have read, concerns the difference between average and incremental cost and the fact that a disease control intervention is the more to be preferred over alternatives, the more readily it can be scaled up or expanded without cost rising so much as to render it a poor bargain. As a program expands to reach more of the susceptible population, incremental costs can go either up or down compared to average cost up to the point where expansion occurs.

Assumption 3 is that "We assume the 'health consumer' is well informed and behaves rationally in making choices." KB do not say who "We" are, but if they mean themselves, they are again utterly wrong about the GBD estimates. The latter make no assumptions at all about how consumers of health care behave, because the GBD is purely and simply an attempt to measure how sick a population is—how much mortality and new disability it suffers in the course of one year—not how the individuals in that population react to their illnesses and injuries. Even the estimates of cost-effectiveness of control programs make no such assumption about the rationality or not of patients and consumers. Of course the degree to which a control intervention is effective will depend, among many other features, on whether the target population has access to the intervention and actually makes use of it. That in turn depends on what they know about both the disease or health problem and the intervention, but that is a matter more of belief and trust than of rationality, which in any case is difficult or impossible to measure. As support for their preposterous assumption, KB cite a paper by DL Schwappach (reference 47), without bothering to look at the publications that spell out the GBD assumptions and check whether that author knows what (s)he is talking about.

What KB cite as Assumption 1, in contrast, is essentially correct: "There is an 'average' disability for each disease state that is the same in all settings." If one does not accept that simplification, there are only two alternatives. One is somehow to know just how bad a disability is, for each and every affected individual (and maybe even whether some people's deaths are worse than others because they are more feared or more excruciating). That is obviously impossible, as an empirical matter—one would need millions or billions more estimates of individual disability weights. That approach would also violate the standard economic assumption that welfare or utility cannot be compared across individuals. It is only by accepting a uniform degree of disability over entire populations that one can work with averages and totals.

Being right about one assumption behind DALY calculations—an assumption that is perfectly clear in the descriptions of those calculations, and therefore not requiring much effort to find, not at all "hidden"—and then being almost hilariously wrong about the other two assumptions does not constitute a good analytical average. Even on the one assumption they got right, KB do not seem to understand why it is a necessary assumption if one is trying to measure how bad a disease state is, rather than some other concept.

The other possibility is to accept averages and totals over large populations but not over the whole world; that is, to impose different disability weights in different places. Since age at incidence and gender are already taken into account in GBD, the only sensible distinction to make would be geographic, and this is what KB insist on for much of the rest of their paper. That is, they maintain that a given disease or condition is worse, say, in sub-Saharan Africa than it is in Europe, because of “local context as a modifier of disease impact”, as discussed on p. 5. The object in assigning disability weights for non-fatal conditions was to judge the intrinsic awfulness of a disease or condition; this is all they are about. It is true that local context can strongly affect how successfully people can cope with a health problem; for example, the loss of one’s legs is easier to bear if one has access to a wheelchair, and streets, sidewalks and buildings have ramps that make navigation easy. But to take that into account is to combine the disease itself with the response to it; GBD estimates are concerned only with the former. Changes to the environment that facilitate coping—such as providing prostheses—are then properly considered to be interventions, and valued accordingly.

The core of KB’s complaint is a paragraph on p. 8. It starts with another utter confusion, in the statement that “Where significant population stratification exists, and risk for disease varies strongly among these strata…there can be no valid global average” (italics in the original). Certainly the risk of any given disease or condition varies strongly from person to person and from place to place: but GBD estimates include only those people who actually die or get sick or hurt. If the risk of illness or death is higher in one population group than another, that will show up in higher incidence; and since GBD is estimated from incidence, the burden of disease will be higher in the group at higher risk. Nothing in that relation requires changing the disability weight (DW) for the people facing more risk. KB seem not to understand that the disability weight describes how bad the condition is for an individual who suffers from it; the burden of that disease or injury in the population is then the product of the disability weight and the incidence (and the duration of the health problem, which depends on age, life expectancy and the average duration of the disease state). Because they do not understand this, they apparently want to make the disability weight do the work of the incidence, being higher where the disease risk is greater.

This confusion about the difference between risks and individual disability weights leads KB to say that “socioeconomic status (SES) modifies or confounds the lifetime risk of acquiring infection, meaning that poor communities jointly suffer more from disease-associated health burden.” Well, yes: in general, poor communities do have a greater burden of disease. But that is entirely accounted for in the way that GBD is estimated. Moreover, the estimated GBD refers to the incidence and duration of disease that occurs, given the amount and kind of preventive, curative and palliative interventions to which a population has access. Thus if “aspects of poverty…restrict access to care”, that will show up in the GBD estimates—either there will be more disease, or it will develop greater severity, or it will last longer than if that population had more access to care. Once again, there is no reason to suppose that disability weights need to differ between the rich and the poor. In particular, there is absolutely no reason to suppose that “the appropriate approach is to…create a weighting system that adjusts the DW for local SES”. To call this “evidence-based adjustment” is an abuse of language; it is just special pleading based on failure to understand what GBD estimates are about and the role of the disability weights in those calculations. Nothing in the way the burden of disease is estimated is biased against the poor, so no correction in their favor is needed

Curiously, KB cite one source (61) in favor of a potentially interesting reason for variation in disability weights—that is, if the disability from a particular disease, schistosomiasis in this case, varies systematically with a person’s age. For this to be relevant, of course, it must be the disability itself that varies, and not the economic consequences, which obviously can differ depending on how old a person is. However, KB do not pursue the question, citing the reference only to argue for taking patients’ (or sufferers’) views into account. In any case, two features of potential age variation need to be understood clearly. First, at any one age, the disability weight would still be the same for all people with the disease; second, variation in the risk of death with age is already accounted for in the GBD estimates. Very young children are more likely than older children or adults to die from malaria, but when they die, the disability weight for morbidity while sick is replaced by the weight of 1.0 for mortality.

To pursue this matter of what GBD does and doesn’t try to measure: the estimates were invented to capture only the health burden from disease and injury—NOT the economic burden, which would require a completely different approach and would vary far more among individuals depending on their age, skills, employment and other factors. Taking those factors into account would mean, for example, that the death of a skilled young adult would count for more than the death of an unskilled older person. As a measure of economic loss, that is unfortunately true—but as a measure of the loss of healthy life, the only difference between the two individuals is the fact that the younger person has more life expectancy left, not that he can earn more.

KB say on p. 5 that “This DALY approach and its ranking tables were believed to provide a more fair comparison of disease burdens, because [it] was believed to be nonsubjective, reflecting social consensus”, and go on to complain (on p. 8) that “the assignment of DWs is, in fact, largely subjective”. That is correct; in fact, it is entirely subjective, for the simple reason that the disutility from being sick or hurt cannot be measured directly. KB do not say who “believed” that because something represents a social consensus, that makes it somehow nonsubjective, but anyone who did so must not have understood one or both of the concepts. Murray and colleagues (reference 9) did indeed aim at “avoiding the potential biases that had been involved in expert assessments of individual diseases”, and that is why the group that developed the disability weights was required to produce weights for all the diseases and conditions in a single exercise. In fact, the genesis of the whole GBD project was the discovery that when individual disease programs in WHO made their own estimates of mortality from “their” diseases, the sum of their estimates greatly exceeded the total estimated deaths in the world.

Anyone may reasonably object to the specific way that WHO derived its estimates of disability weights, including dependence on “highly educated individuals” who could understand the person trade-off (PTO) method used to elicit relative weights. It may also be reasonable to be concerned that when that exercise is repeated in different countries, the people whose views are solicited give different rankings. What is not reasonable is to suppose that because such different views exist—a consequence of the necessarily subjective nature of the rankings—the WHO estimates are wrong. It is also important to remember that some of the views elicited in other populations may be morally repugnant. For example, men in much of the world systematically value women less than men, so they would presumably assign lower disability weights for conditions suffered mostly or exclusively by women, particularly problems of pregnancy and childbirth. WHO tried to keep its exercise free of such prejudices, a goal probably helped by asking the views of more highly educated people. KB’s answer to this problem is that the consensus assignment of weights probably “reflected the individual and cultural biases of the panels and of the DW facilitators themselves”, and go on to ask for “additional validation”. But how is one to validate a subjective consensus ? One can ask more, and more varied, individuals their opinions, as Ustun and colleagues (reference 23) did, but that will generally only broaden the range of disagreement and introduce different sets of biases, some of which will be ethically unacceptable. The subjectivity of these estimates is intrinsic to the effort to take account of both mortality and disability in the GBD. If a non-fatal condition is agreed to be worse than perfect health but not so bad as death, then it must be assigned a weight somewhere between the extremes of zero and 1.0. All the argument is about whose views should be consulted for that purpose.

KB are wrong again, concerning one specific feature of the GBD calculations, claiming (p. 5) that “no truly substantive changes have been made to the DALY system” since it was introduced. In fact, the original GBD estimates, referring to 1990 and published in 1996, included an age-weighting factor that started at zero, rose to a peak at age 25 and then declined almost exponentially. That corresponds to K =1 in the formulas on pp. 3-4. KB appear not to know that in the more recent estimates referring to 2001, age-weighting was abandoned. The logic of this truly substantive change is straightforward. Giving different intrinsic values to life at different ages makes considerable sense initially, going up from birth, because an infant is simply not as complete a person as even a two- or three-year-old child. However, it is not justified to value a year of life at age 40 as worth less than a year at age 20. (A death at age 40 causes a loss of fewer DALYs than a death at 20, purely because of the difference in remaining life expectancy, not because of any difference in the valuation of each year of life. Discounting the future shrinks that difference, because some of the years lost by the 20-year-old will occur farther in the future than any years lost by the 40-year-old.) KB attack age-weighting on p. 14, and then say (p. 18) that “Age-weighting should not be used to value health outcomes”, basing their case on differences between rich and poor societies in the ages at which people start and stop working. Again, this confuses the estimate of how bad it is to be sick with the economic consequences of illness, KB’s consistent misinterpretation of what the GBD is about. The authors of the most recent estimates of GBD agree that age-weights are a bad idea, but for the more general reason of how life should be valued, not economic productivity. KB never got around to reading the 2006 publication of disease burden estimates, which is not among their many references.

KB title their paper, “Why Global Burden of Disease Valuations Underestimate the Burden of Neglected Tropical Diseases”. Up to this point, the discussion here applies to all the diseases and conditions included in the GBD; what has any of this got to do specifically with NTDs ? KB appear to think that GBD systematically underestimates the burden of chronic diseases (including NTDs) relative to acute health problems, but there is no basis for that view. Since burden estimates take account of the whole future stream of ill health, long-lasting conditions contribute much more, for a given disability weight, than disease states that resolve quickly. The only reason to suppose that chronic diseases are undervalued is that the future is discounted—but that affects mortality also, and discounting makes excellent economic sense. One can argue whether 3% per year is the right discount rate, but that question has no right answer. Just like the disability weights, the discount rate in GBD is a consensus estimate, not a measurable or verifiable number.

KB also note (p. 5) that sufferers from NTDs often also have co-morbidities or concurrent infections, and those ought somehow to be taken into account. GBD estimates do not do that, because of two serious problems. One is the total disability from having two or more health problems at once is not necessarily the sum of the separate disabilities, which might in fact exceed 1.0 and appear to be worse than death. The difficulties of reaching consensus on the appropriate weights for more than 100 disease states would be multiplied many times over if respondents had to consider and assign weights for every non-trivial combination of two or more diseases. If two conditions were always found together, clearly there should be a disability weight for their combined effect; but when condition B only sometimes occurs together with condition A, there is no reason for increasing the disability weight of condition A by itself. The other reason co-morbidities are not considered is, once again, the limitation of information—to take them into account, one would have to know not only how many people have TB and how many have musculoskeletal problems, but how many people suffer both problems. The information requirements are daunting enough, taking diseases one at a time.

The argument that GBD systematically underestimates the importance of NTDs does not hold water; in fact, it is special pleading of exactly the sort that the burden of disease exercise was meant to overcome. Part of it comes from confusing disease burden with economic consequences; part of it comes from confusion about the roles of disability weights and their relation to incidence and duration; much of it comes simply from the authors’ relying on the opinions of many other critics of how DALYs are constructed, rather than carefully studying the basic documents where the assumptions and limitations are spelled out. Vigorous debate over whether the global burden of disease adequately represents the state of the world’s health is welcome, in fact necessary for improved estimates. Such debate should, however, be based on a much more thorough comprehension of what the GBD does and does not attempt to do. If understanding and debate lead to revision of the estimates—as they have already done where age-weights are concerned—the result should be better measures of global health. No amount of debate, though, will ever eliminate the subjective nature of many of the assumptions that must be made, nor will it ever provide objective validation: at best, the consequence will be a wider consensus and a clearer understanding.


Philip Musgrove
19 July 2008

The author was one of the authors of the World Development Report: Investing in Health (World Bank, 1993), editor in chief of the World Health Report 2000—Health Systems: Improving Performance (World Health Organization, 2000) and one of the editors of Disease Control Priorities in Developing Countries, 2nd edn (Oxford University Press for the World Bank, 2006).