Reader Comments

Post a new comment on this article

Evaluating the performance of antiretroviral treatment programs: mortality and loss to follow-up

Posted by plosmedicine on 31 Mar 2009 at 00:18 GMT

Author: Till Bärnighausen
Position: Associate Professor of Health and Population Studies
Institution: Africa Centre for Health and Population Studies, University of KwaZulu-Natal, and Department of Population and International Health, Harvard School of Public Health, Boston
Additional Authors: Victoria Xolo (KwaMsane ART Clinic, and Africa Centre for Health and Population Studies), Graham S. Cooke (Africa Centre for Health and Population Studies, and Faculty of Medicine, Imperial College, London)
Submitted Date: December 12, 2007
Published Date: December 28, 2007
This comment was originally posted as a “Reader Response” on the publication date indicated above. All Reader Responses are now available as comments.

The study by Rosen et al. [1] has received extensive media coverage, is timely, and should be welcomed. We caution, however, against using "retention" as defined in the study to evaluate the performance of antiretroviral treatment (ART) programs. The authors calculate "retention" as unity minus "attrition", where "attrition" is the proportion of patients who were lost to follow-up, died or stopped ART while in care. "Across all the cohorts," they report, "the largest contributor to attrition was loss to follow-up (56% of attrition), followed by death (40% of attrition)." The authors then use "retention" to compare the performance of ART programs: "Of those reporting 24 mo of follow-up, the best program retained 85% of patients and the worst retained 46%."

Such comparisons are problematic for several conceptual reasons. For one, "retention" weighs equally two very different outcomes of ART: death and loss to follow-up (LTFU). Thus, two ART programs with 50% retention after 2 years which are equal in all aspects except that 50% of patients in the first program have died, while 50% of patients in the second program are lost to follow-up, will have performed equally well according to the "retention" criterion. In most circumstances, however, death would be considered a worse outcome than LTFU and the second program should be preferred over the first.

Moreover, neither mortality nor LTFU on its own is an ideal measure to compare program performance because neither controls for patient heterogeneity. ART program managers can decrease average mortality by improving quality of care or by excluding the sickest patients from ART; and they can decrease LTFU by increasing patient satisfaction or by barring patients with high antecedent probability of LTFU from ART.

First, imagine two programs equal in all aspects except that the first program excludes patients with CD4 counts less than 50/µl from ART, while the second one does not. The first program will be preferred to the second under the "retention" criterion because mortality on ART is highest in the group with baseline CD4 counts less than 50/µl. While exclusion of patients with low CD4 counts from ART has been suggested on cost-effectiveness grounds, many programs in Africa, including the one in which we work, attempt more rapid initiation of patients with CD4 count less than 50/µl, as the mortality reduction due to ART is very large in this group.

Second, imagine two programs, equal in all aspects except that one, but not the other, excludes people from ART who display characteristics indicating high probability of LTFU. For instance, the Médecins Sans Frontières (MSF) ART program in Khayelitsha, South Africa, required for ART eligibility "a record of regular clinic attendance" and a "commitment not to move away from the township", and excluded from ART patients who abused alcohol or suffered from "untreated active depression" [2]. If such criteria predict future LTFU, programs that apply them will be ranked as better performers under the "retention" criterion than programs that do not.

Prioritization of patients entails many difficult choices – such choices should be made in open deliberations that follow the principles of a fair process [3] and should not be forced on ART programs by inadequate measures of performance. Rather than cross-program comparison of performance, it may be more useful to evaluate a program’s impact on health outcomes and social and economic well-being within the population it serves.

1. Rosen S, Fox MP, Gill CJ (2007) Patient retention in antiretroviral therapy programs in sub-saharan Africa: a systematic review. PLoS Med 4: e298.

2. Fox RC, Goemaere E (2006) They call it "patient selection" in Khayelitsha: the experience of Médecins Sans Frontières-South Africa in enrolling patients to receive antiretroviral treatment for HIV/AIDS. Camb Q Healthc Ethics 15: 302-312.

3. Daniels N (2004) How to achieve fair distribution of ARTs in 3 by 5: fair process and legitimacy in patient selection. Geneva: WHO.

No competing interests declared.