Advertisement
  • Loading metrics

Toward Measuring Schistosoma Response to Praziquantel Treatment with Appropriate Descriptors of Egg Excretion

  • Piero L. Olliaro ,

    olliarop@who.int

    Affiliations UNICEF/UNDP/World Bank/WHO Special Programme on Research and Training in Tropical Diseases (TDR), World Health Organization, Geneva, Switzerland, Centre for Tropical Medicine, Nuffield Department of Medicine, University of Oxford, Oxford, United Kingdom

  • Michel Vaillant,

    Affiliation Centre of Competence for Methodology and Statistics (CCMS), Luxembourg Institute of Health (LIH), Strassen, Luxembourg

  • Aïssatou Diawara,

    Affiliation Department of Biology, Division of Science and Mathematics, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates

  • Jean T. Coulibaly,

    Affiliations Unité de Formation et de Recherche Biosciences, Université Félix Houphouët-Boigny, Abidjan, Côte d’Ivoire, Centre Suisse de Recherches Scientifiques en Côte d’Ivoire, Abidjan, Côte d’Ivoire, Department of Medical Parasitology and Infection Biology, Swiss Tropical and Public Health Institute, Basel, Switzerland, University of Basel, Basel, Switzerland

  • Amadou Garba,

    Affiliations University of Basel, Basel, Switzerland, Department of Epidemiology and Public Health, Swiss Tropical and Public Health Institute, Basel, Switzerland, Réseau International Schistosomoses, Environnement, Aménagement et Lutte (RISEAL-Niger), Niamey, Niger

  • Jennifer Keiser,

    Affiliations Department of Medical Parasitology and Infection Biology, Swiss Tropical and Public Health Institute, Basel, Switzerland, University of Basel, Basel, Switzerland

  • Charles H. King,

    Affiliations Center for Global Health and Diseases, Case Western Reserve University, Cleveland, Ohio, United States of America, Schistosomiasis Consortium for Operational Research and Evaluation, University of Georgia, Athens, Georgia, United States of America

  • Stefanie Knopp,

    Affiliations University of Basel, Basel, Switzerland, Department of Epidemiology and Public Health, Swiss Tropical and Public Health Institute, Basel, Switzerland, Wolfson Wellcome Biomedical Laboratories, Department of Life Sciences, Natural History Museum, London, United Kingdom

  • Aly Landouré,

    Affiliation Institut National de Recherche en Santé Publique, Bamako, Mali

  • Eliézer K. N’Goran,

    Affiliations Unité de Formation et de Recherche Biosciences, Université Félix Houphouët-Boigny, Abidjan, Côte d’Ivoire, Centre Suisse de Recherches Scientifiques en Côte d’Ivoire, Abidjan, Côte d’Ivoire

  • Giovanna Raso,

    Affiliations University of Basel, Basel, Switzerland, Department of Epidemiology and Public Health, Swiss Tropical and Public Health Institute, Basel, Switzerland

  • Alexandra U. Scherrer,

    Affiliation Division of Infectious Diseases and Hospital Epidemiology, University Hospital Zurich, University of Zurich, Zurich, Switzerland

  • José Carlos Sousa-Figueiredo,

    Affiliation Department of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine, London, United Kingdom

  • Katarina Stete,

    Affiliations University of Basel, Basel, Switzerland, Department of Epidemiology and Public Health, Swiss Tropical and Public Health Institute, Basel, Switzerland, Center for Infectious Diseases and Travel Medicine, Department of Medicine, University Hospital Freiburg, Freiburg im Breisgau, Germany

  • Xiao-Nong Zhou,

    Affiliations National Institute of Parasitic Diseases, Chinese Center for Disease Control and Prevention, Shanghai, People’s Republic of China, Key Laboratory of Parasite and Vector Biology of the Chinese Ministry of Health, WHO Collaborating Center for Malaria, Schistosomiasis and Filariasis, Shanghai, People’s Republic of China

  •  [ ... ],
  • Jürg Utzinger

    Affiliations University of Basel, Basel, Switzerland, Department of Epidemiology and Public Health, Swiss Tropical and Public Health Institute, Basel, Switzerland

  • [ view all ]
  • [ view less ]

Toward Measuring Schistosoma Response to Praziquantel Treatment with Appropriate Descriptors of Egg Excretion

  • Piero L. Olliaro, 
  • Michel Vaillant, 
  • Aïssatou Diawara, 
  • Jean T. Coulibaly, 
  • Amadou Garba, 
  • Jennifer Keiser, 
  • Charles H. King, 
  • Stefanie Knopp, 
  • Aly Landouré, 
  • Eliézer K. N’Goran
PLOS
x

Abstract

Background

The control of schistosomiasis emphasizes preventive chemotherapy with praziquantel, which aims at decreasing infection intensity and thus morbidity in individuals, as well as transmission in communities. Standardizing methods to assess treatment efficacy is important to compare trial outcomes across settings, and to monitor program effectiveness consistently. We compared customary methods and looked at possible complementary approaches in order to derive suggestions for standardizing outcome measures.

Methodology/Principal Findings

We analyzed data from 24 studies conducted at African, Asian, and Latin American sites, enrolling overall 4,740 individuals infected with Schistosoma mansoni, S. haematobium, or S. japonicum, and treated with praziquantel at doses of 40–80 mg/kg. We found that group-based arithmetic and geometric means can be used interchangeably to express egg reduction rates (ERR) only if treatment efficacy is high (>95%). For lower levels of efficacy, ERR estimates are higher with geometric than arithmetic means. Using the distribution of individual responses in egg excretion, 6.3%, 1.7% and 4.3% of the subjects treated for S. haematobium, S. japonicum and S. mansoni infection, respectively, had no reduction in their egg counts (ERR = 0). The 5th, 10th, and 25th centiles of the subjects treated for S. haematobium had individual ERRs of 0%, 49.3%, and 96.5%; the corresponding values for S. japonicum were 75%, 99%, and 99%; and for S. mansoni 18.2%, 65.3%, and 99.8%. Using a single rather than quadruplicate Kato-Katz thick smear excluded 19% of S. mansoni-infected individuals. Whilst the effect on estimating ERR was negligible by individual studies, ERR estimates by arithmetic means were 8% lower with a single measurement.

Conclusions/Significance

Arithmetic mean calculations of Schistosoma ERR are more sensitive and therefore more appropriate to monitor drug performance than geometric means. However, neither are satisfactory to identify poor responders. Group-based response estimated by arithmetic mean and the distribution of individual ERRs are correlated, but the latter appears to be more apt to detect the presence and to quantitate the magnitude of suboptimal responses to praziquantel.

Author Summary

To identify whether a person is infected with parasitic worms, stool or urine samples are examined for worm eggs. The drug praziquantel is used against the parasitic disease schistosomiasis. However, there is no definitive agreement as to how the efficacy of praziquantel is best expressed. We put together a database from various studies of the efficacy of praziquantel against schistosomiasis. Efficacy was measured using customary methods: cure rate (CR: percentage of people with eggs in their stool/urine before treatment who became egg-negative after treatment); and egg reduction rate (ERR; percentage reduction in the number of eggs in the stool/urine after treatment, where the mean number of eggs from all people treated is calculated using either geometric or arithmetic means). We found that arithmetic and geometric means can be used interchangeably only if treatment efficacy is very high; arithmetic means are more sensitive to capture drops in efficacy expressed by ERR. A valid complement for drug efficacy monitoring is to study the distribution of individual responses in egg excretion that allows identifying in a single measure both those who had an adequate response to treatment and those who respond less well; e.g., the 5% of the patients with the lowest ERRs.

Introduction

Schistosomiasis is a parasitic disease caused by blood flukes of the genus Schistosoma. The three main species infecting humans are S. haematobium (causing urogenital schistosomiasis), S. japonicum, and S. mansoni (the latter two responsible for intestinal schistosomiasis) [1]. The backbone of the global strategy for controlling the morbidity caused by schistosomiasis is the periodic administration of single-dose oral praziquantel (usually given at 40 mg/kg body weight). This strategy is termed “preventive chemotherapy”, whereby praziquantel is administered without prior diagnosis [2,3] to entire communities or target groups, most importantly school-aged children [46], depending on the level of endemicity [7,8].

How efficacy of antischistosomal drugs (and anthelmintic treatments in general) should be measured has been, and still is, a matter of debate in the research and disease control communities. One limitations of the current treatment outcome measure–i.e., parasite egg excretion–is that it is a proxy for drug effects on adult worms, which could also be confounded by various factors, including facultative temporary cessation of excretion by the adult worm [3]. Other, more direct but not yet widely used methods of worm vitality are the detection of specific antigens, like the circulating cathodic antigen (CCA) and the circulating anodic antigen (CAA) [9,10]. Clinical trials have traditionally used cure rates (CRs) as the main drug efficacy endpoint, and expressed results as the proportion of infected individuals who convert to a negative stool or urine sample post-treatment [1113]. However, the World Health Organization (WHO) recently issued guidelines that recommend the egg reduction rate (ERR) as the primary outcome measure, especially when assessing programmatic treatment effectiveness [3]. This entails a quantitative diagnostic test based on the microscopic detection and enumeration of parasite eggs in small amounts of stool (usually 41.7 mg) and the estimation of the number of eggs per 1 g (EPG) of feces (S. japonicum or S. mansoni) or per 10 ml of urine (S. haematobium). The ERR measures the overall effect of treatment on the entire group of infected subjects treated (ignoring individual variability) and is expressed as the ratio between the mean of the pre- and post-treatment egg counts [3]. ERR is considered more suitable than CR to assess the impact of preventive chemotherapy on morbidity (which is commensurate to infection intensity) in the context of continuous risk of reinfection and in view of the low sensitivity of current diagnostic methods [12,14].

Which type of means (e.g., geometric mean (GM) or arithmetic mean (AM)) should be used to express treatment outcomes against helminthiases at the community level is an additional subject of debate [15,16]. Thus far, studies of treatment efficacy have predominantly reported results using GM egg counts [15] but recently, the use of AM egg counts has been advocated [17,18]. The issue is that egg counts are not normally distributed, even after logarithmic transformation, which would call for using GM [19]. However, GM hide extreme values (e.g., (i) a small proportion of individuals disproportionally contributing to total egg excretion, and (ii) individuals who do not respond to treatment), which are important when assessing the effects of interventions, and which are better captured by using AM.

A further complication when dealing with different studies is the diversity in methodologies, in particular: (i) which diagnostic method is used (e.g., single or multiple urine filtration for detection of S. haematobium eggs) [20] or the Kato-Katz technique for detection of S. japonicum or S. mansoni eggs in fecal samples, whose sensitivity depends upon the baseline infection intensity, the number of thick smears from a single sample, and the number of stool specimens examined [2124]; and (ii) how many weeks post-treatment effects are measured, which also depends on Schistosoma species [25,26].

The objectives of this paper were to compare customary methods to assess the efficacy of praziquantel for treating schistosomiasis and to explain differences; to identify possible alternative approaches to express treatment effects on egg excretion; and to verify whether the size of treatment effects for intestinal schistosomiasis change when measured with a single or quadruplicate Kato-Katz thick smears. The overall aim of these analyses was to derive suggestions for standardizing outcome measures in future drug efficacy studies. For this purpose, we combined and analyzed available data from various studies where praziquantel was used to treat infections with different Schistosoma species.

Methods

Ethics Statement

All studies selected for the current secondary analyses had been approved by the relevant institutional review boards and ethics committees, and were conducted according to international ethics standards (for details, see individual publications [2536]). Data received from the individual studies were completely anonymized.

Datasets Analyzed

We built a common database from 24 studies including 4,740 individuals assigned to three different treatment groups who had received either 40 mg/kg (18 studies, 3,713 individuals), 60 mg/kg (five studies, 690 individuals), or 80 mg/kg (one study, 337 individuals) praziquantel against S. haematobium [25,2729,34,36], S. japonicum [31,32], and S. mansoni [21,23,2628,30,31,33,35].

The main study characteristics, including Schistosoma species, praziquantel dose, age of participants, time-point of treatment follow-up, and diagnostic approach, are summarized in Table 1. Studies enrolled a total of 4,740 individuals, of whom 2,633 (55.5%) were infected with S. haematobium, 1,804 (38.1%) with S. mansoni, and the remaining 303 (6.4%) with S. japonicum. Studies generally enrolled children and adolescents except one study in the People’s Republic of China [32] and another one in Côte d’Ivoire [33]. The praziquantel dose was 40 mg/kg in 17 studies, 60 mg/kg in five studies, and 80 mg/kg in one study. Follow-up was within 3 weeks in 13 studies, four weeks in two studies, within 2 months in six studies, and longer in the remaining three studies.

For the detection of S. haematobium infection, two diagnostic approaches were employed: (i) a single urine filtration slide in five studies (one of them was carried out on the same sample but at two different time points) [25,26,28,29,34]; and (ii) duplicate urine filtration slides in one study [36]. For the diagnosis of S. japonicum, duplicate Kato-Katz thick smears from each of two stool specimens were subjected to microscopic examination in all studies. For the detection of S. mansoni, the most common diagnostic approach was duplicate Kato-Katz thick smears from each of two stool specimens. In one study, a single Kato-Katz thick smear was performed on five samples at baseline and triplicate Kato-Katz thick smears for four samples at follow-up [35,37].

Efficacy Outcomes and Calculations

Treatment response was assessed both at the overall group and the individual level. The AM and GM EPG values were calculated at pre- and post-treatment for S. mansoni and S. japonicum by multiplying the individual fecal egg counts (FECs) obtained by a single Kato-Katz thick smear (41.7 mg) by a factor of 24. For S. haematobium, egg counts are presented as eggs per 10 ml of urine. Drug efficacy was expressed as ERR and CR.

ERR (arithmetic (ERRAM) or geometric (ERRGM)) was calculated as the ratio of the difference between the (arithmetic or geometric) means of the pre- and post-treatment EPG or eggs per 10 ml urine to the pre-treatment (arithmetic or geometric) mean EPG or eggs per 10 ml urine: ERR = [(mean egg countpre-treatment − mean egg countpost-treatment) / mean egg countpre-treatment] x 100

GM egg counts were calculated as follows: , where xijk is the observed egg count for individual host i, Schistosoma species j, and study k; njk is the number of hosts who provided a (pre- and post-treatment) sample for determination of infection intensity for each Schistosoma species and study, and c is a constant added to each count to allow inclusion of zero counts (negative test) [38]. Confidence intervals (CIs) for the ERR (calculated with AM and GM) were determined by using a bootstrap resampling method (with replacement) over 1,000 replicates and expressed as a univariate calculation of the 2.5th and 97.5th percentiles.

Individual ERR was calculated as the ratio of the difference between the pre- and post-treatment EPG or eggs per 10 ml urine to the pre-treatment EPG or eggs per 10 ml urine multiplied by 100.

CRs and 95% binomial CIs were the percentage of infected individuals negative for Schistosoma (in their urine or stool) at post-treatment follow-up. The distribution of individual responses in egg excretion was categorized as (i) negative (corresponding to CR), (ii) reduction, (iii) no change or increase, and further expressed in centiles to quantitate the fraction of poor responders.

Comparing Methods of Assessing Drug Efficacy

We compared (i) the ERRAM versus ERRGM and (ii) the CR versus ERR. The results were presented graphically in modified L’Abbé plots with 95% CIs for both comparisons, and additionally in Bland and Altman plots for ERR using AM and GM. The coefficient of determination (R2) was also calculated.

A linear mixed model of the difference of the ERRs calculated as GM and AM (ΔERRg,a) was developed to estimate which parameter could better predict the difference between the two ERR calculations, with the average of the two mean ERRs (GM and AM) set as covariate. Such a model was extrapolated from the Bland and Altman regression by further including predictive factors. In order to evaluate the effect of different factors on ΔERRg,a, the ERR was calculated on the different strata defined by the combination of the categories of the following parameters: age, sex, treatment dose, and Schistosoma species. The same age categories were defined across all studies. The linear mixed model was estimated including these parameters as independent factors. 95%CIs of the difference between the two ERR calculations were calculated by using Tango’s score confidence interval which was shown in the literature to outperform other calculation methods in the case of correlated proportions [39,40].

Modeling was carried out through a shrinkage method of variable selection. Variables were first selected using the ElasticNet procedure, which is mixing a least absolute shrinkage and selection operator (LASSO) procedure and ridge regression [41]. Subsequently, a variance-covariance matrix structure was selected among unstructured, variance components, autoregressive, compound symmetry and Toeplitz structures, which minimized the Aikake information criterion corrected (AICC) for finite sample size. Post-hoc tests on each parameter were carried out with a Tukey adjustment. Pairwise differences in least square means (LSM) were calculated for fixed values of the average of the AM and GM ERRs (for the range 70–99%), thereby evaluating the influence of exogenous parameters on the bias between the two ERRs. This bias was evaluated in the Bland and Altman method by regressing the difference of the two methods by their mean.

All tests were two-tailed; a p-value of 5% was deemed significant. Only studies with treatment follow-up examination done within 90 days were included in the models (to minimize the effect of reinfection after praziquantel administration). Calculations and analyses were performed by using SAS system version 9.3 (SAS Institute, Cary, NC, United States of America).

Comparing Efficacy Assessments Based on Single versus Quadruplicate Kato-Katz Thick Smears for S. mansoni

For S. mansoni, we compared single (using the first thick smear on the first stool specimen) versus quadruplicate (using four thick smears of the same stool specimen) Kato-Katz thick smears for expressing CR and ERR (with AM and GM). In this sub-analysis, we only included individuals infected with S. mansoni whose first slide on the first fecal sample was positive.

Within this population, we calculated and compared the overall AM and GM pre- and post-treatment FECs and the respective ERRs and CRs based on single and quadruplicate Kato-Katz thick smears with 95% CIs calculated by boot-strapping. The results of the individual studies were presented graphically in Bland and Altman plots and in modified L’Abbé plots with 95% CIs, and the coefficient of determination (R2) was calculated.

Results

Study Characteristics

The distributions of the raw egg counts at baseline by Schistosoma species for each study (including AMs and GMs) are presented in Fig 1. Efficacy outcomes were analyzed on a total of 4,375 individuals with pre- and post-treatment egg counts. Among them, 2,365 (54.1%) were infected with S. haematobium, 1,708 (39.0%) with S. mansoni, and the remaining 300 (6.9%) with S. japonicum. Details of baseline FECs and drug efficacy outcomes, including group-based means and individual responses by study and Schistosoma species, are presented in Table 2.

thumbnail
Fig 1. Distribution of raw egg count at baseline by Schistosoma species and study.

A. Schistosoma haematobium. B. Schistosoma mansoni. C. Schistosoma japonicum.

https://doi.org/10.1371/journal.pntd.0003821.g001

thumbnail
Table 2. Summary of study outcomes, group-based mean responses, and individual responses by study and species.

https://doi.org/10.1371/journal.pntd.0003821.t002

Comparing ERRs Obtained with AM versus GM

ERRAM ranged from 17.0% to 99.8% and ERRGM from 50.7% to 99.8% for S. haematobium; 81.5–100% (ERRAM) and 95.9–100% (ERRGM) for S. mansoni; 83.5–99.9% (ERRAM) and 86.9–99.9% (ERRGM) for S. japonicum. The 95% CIs estimated by boot-strapping tended to be wider with AM compared to GM. Among the 24 studies included in our analyses, six had an ERR <90% by AM (two if restricted to studies with 3 weeks’ follow-up), but only two by GM.

The modified L’Abbé plot (Fig 2A) indicates that ERRs tend to be higher when calculated using GM. The R2 of the linear regression showed a strong linear correlation for S. japonicum (R2 = 1.00, three studies) and S. haematobium (R2 = 0.88, eight studies), but a weaker linear correlation for S. mansoni (R2 = 0.46, 13 studies).

thumbnail
Fig 2. Comparison of egg reduction rate calculated as arithmetic mean (ERRAM) versus geometric mean (ERRGM), both with estimated 95% confidence intervals (CIs).

A. L’Abbé plot, B. Bland & Altman plot. For each point, the 95% CIs are shown in the L’Abbé plot, where the red dashed diagonal line of no difference between the ERR calculated with both metrics. The blue dashed line represents the threshold for acceptable efficacy defined by WHO [3].

https://doi.org/10.1371/journal.pntd.0003821.g002

Modeling Group-Based ERRs

The mixed linear model found a significant relationship between the difference between ERRAM and ERRGM and their mean value. However, introducing the variables identified as predictors by the ElasticNet procedure, coupled with model averaging based on 1000 replicates (sampling with replacement)–mean baseline epg, year of study and species–rendered this relation non-significant (meaning that none of these covariates could explain the differences between ERRAM and ERRGM.)

LSM pairwise comparisons and ERR (individually for AM and GM) models showed a significantly better consistency between ERRs calculated with AM and GM for S. haematobium than for S. mansoni. Study participants’ age was found to have an effect only for ERRGM (higher for school-aged children and adults than for preschool-aged children; S1 Table).

Distribution of Individual Treatment Responses in Egg Excretion

Group means and individual responses are presented in Table 2. Individual responses are also displayed graphically in Fig 3 (panel A as bar diagrams; panel B as centile distributions). Overall, 6.3% (ranging in individual studies from 0 to 39%), 1.7% (0–5%); and 4.3% (0–11%) of the subjects treated for S. haematobium, S. japonicum, and S. mansoni infection, respectively, had no reduction in their egg counts (ERR = 0). The 5th, 10th, and 25th centiles of the subjects treated for S. haematobium had individual ERRs of 0% (ranging in individual studies from 0 to 99.7%), 49.3% (0–100%), and 96.5% (0–100%); the corresponding values for S. japonicum were 75.0% (0–97.4%), 99.0% (25.6–98.8%), and 99.0% (96.8–99%), and for S. mansoni 18.2% (0–97.4%), 65.3% (0–99.4%), and 99.8% (0–99.8%).

thumbnail
Fig 3. Distribution of individual ERR.

A. Bar diagrams, B. Centile distribution.

https://doi.org/10.1371/journal.pntd.0003821.g003

The centile distribution of iERRs in studies with ERRAM <90% was shifted to the right and clearly distinct from those with ERRAM ≥90% (Fig 4).

thumbnail
Fig 4. Centile distribution of individual ERRs by species for all studies (A) and for studies with assessment within 28 days divided by ERRAM ≥90% and <90% (B).

https://doi.org/10.1371/journal.pntd.0003821.g004

When considering only studies assessing outcomes within a maximum of 28 days (13 studies evaluated drug efficacy at 21 days, two at 28 days), only two (both treating S. mansoni with 40 mg/kg praziquantel with 21-day follow-up [27,30]) had an ERRAM <90%; in terms of individual responses, the 30th and 36th centile, respectively had ERRs <90%, and 10.8% and 7.1% of patients, respectively had no change in their egg counts (ERR = 0). Three additional studies, all with ERR AM >90%, had individual ERRs <90% in the 13th centile: one study on S. haematobium treated with 40 mg/kg and 21-day follow-up [27], and two studies on S. mansoni treated with 40 or 60 mg/kg with 28-day follow-up [35,37]. In these studies, 3.2%, 3.5% and 2.4%, respectively of individuals had no decrease in egg counts. Across these studies, ERRAM and iERR were highly correlated (R2 0.95).

Comparing CR versus ERR

For all three Schistosoma species, the CR was systematically lower than ERR, regardless of whether AM or GM was employed, except in a single study [29]. ERR >90% corresponded to CRs ranging from 51.4% to 99%. Only when ERRs were very high (range: 97.7–100%) there was a good agreement between both indicators. The CR ranged from 82.1% to 100% (Fig 5).

thumbnail
Fig 5. Comparison between egg reduction rate (ERR) and cure rate (CR).

The red dashed diagonal line of no difference between the ERR calculated with both metrics.

https://doi.org/10.1371/journal.pntd.0003821.g005

Multiple versus Single Kato-Katz Thick Smear Examination for S. mansoni

Among the 1,435 individuals enrolled in the studies who were found positive for S. mansoni eggs in their stool based on quadruplicate Kato-Katz thick smears, 1,167 (81.3%) were diagnosed positive on the first Kato-Katz thick smear. In this subset, we found that the results in the individual studies were highly correlated (Fig 6A–6B; R2 = 0.95 for AM and R2 = 0.86 for GM). The same number of studies (n = 2) had AM ERR of less than 90% with either approach.

thumbnail
Fig 6. Comparison of egg reduction rate (ERR) calculated from a single versus quadruplicate Kato-Katz thick smears, both with estimated 95% confidence intervals (CIs).

A. L’Abbé plot. For each point, the 95% CIs are shown. The red dashed diagonal line of no difference between the ERR calculated with both metrics. The blue dashed line represents the threshold for acceptable efficacy defined by the WHO [3]. B. Bland-Altman plot. The red dashed lines show the interval that determines the agreement of the two metrics, set at 5%. The blue dashed line represents the threshold for acceptable efficacy defined by the WHO [3].

https://doi.org/10.1371/journal.pntd.0003821.g006

On aggregate, the FECs at baseline were lower with quadruplicate than with a single Kato-Katz thick smear when using either AM (1,046 EPG versus 2,617 EPG) or GM (342 EPG versus 575 EPG); the ERR estimates were comparable only by GM (99.5% versus 99.8%) whereas by AM, the estimate was lower with quadruplicate than single examinations (86.9% versus 94.9%) (Table 3). In order to verify if using a single Kato-Katz thick smear selected for a different sample, we compared the baseline FECs and ERRs assessed with quadruplicate Kato-Katz thick smears on the overall sample of these studies (n = 1,435) to that of this subgroup with the first Kato-Katz thick smear positive (n = 1,167): while the overall average baseline FECs was approximately threefold lower for the latter, there was no difference in ERR between the two groups for either AM and GM.

thumbnail
Table 3. Data outcomes at baseline in individual positives on single or quadruplicate Kato-Katz thick smears.

https://doi.org/10.1371/journal.pntd.0003821.t003

In contrast to ERRs, CRs derived from single versus quadruplicate Kato-Katz thick smears were poorly correlated (R2 = 0.50) (Fig 7). With few exceptions, the CR assessed from single Kato-Katz thick smear (CR = 92.0%) was consistently higher than that from quadruplicate Kato-Katz thick smears (CR = 82.6%) in the individual studies and the difference between the two approaches increased with decreasing efficacy (Fig 7A and 7B). The overall CR was 88.0% applying a single and 79.0% using quadruplicate Kato-Katz thick smears (Table 4).

thumbnail
Fig 7. Comparison of cure rate (CR) calculated from a single versus quadruplicate Kato-Katz thick smears, both with estimated 95% confidence intervals (CI).

A. L’Abbé plot. For each point, the 95% CIs are shown. The red dashed diagonal line of no difference between the ERR calculated with both metrics. B. Bland-Altman plot. The red dashed lines show the interval that determines the agreement of the two metrics, set at 5%.

https://doi.org/10.1371/journal.pntd.0003821.g007

thumbnail
Table 4. Drug efficacy outcomes at baseline in individuals detected positive for S. mansoni by a single or quadruplicate Kato-Katz thick smear examinations at baseline.

https://doi.org/10.1371/journal.pntd.0003821.t004

Discussion

Preventive chemotherapy with praziquantel is the current backbone of the global strategy to control the morbidity caused by schistosomiasis in high-endemicity areas [2,4,6,8]. Monitoring of praziquantel efficacy should accompany schistosomiasis control programs in order to identify promptly suboptimal responders; to that effect, the WHO has issued standard procedures for control programs based on one single measurement around 3 weeks post-treatment [3]. At the same time, more work is needed to improve the current evidence-base for decision-making: in order to provide reliable information, it is important to agree on a robust statistical approach to assess drug efficacy, especially in clinical trials, and to use standardized, quality-controlled diagnostic methods that are comparable from one setting to another [18,42]. To date, both CR and ERR (the latter based on either AM or GM), are used for assessing drug efficacy. The stool and urine sampling and diagnostic approaches vary across studies, and these issues have important ramifications for drug efficacy estimates.

The controversy over the use of AMs or GMs to measure anthelmintic treatment efficacy as assessed by ERR in parasitic nematodes of cattle, and more recently in those of humans, has already been expounded in several studies, with contrasting results [43] favoring either AMs [17], or GMs (see e.g. [15]) or pointing to inadequacies of both [44]. The overall aim of this study was to compare and contrast different approaches to express treatment effects on Schistosoma infections in order to derive indications for standardizing future studies of drug efficacy.

The first specific objective of this paper was to compare customary measures of drug efficacy. We first considered whether efficacy assessment based on ERR changes when using AM or GM egg counts for the three predominant human Schistosoma species. While both means were in the same range for all species and showed a moderate level of correlation, the discrepancy between AM and GM became wider with decreasing drug efficacy. As previously reported by some [17,43,45]] but not all authors[17], GM estimates tended to be higher than AM. These findings suggest that the two means can be used interchangeably if drug efficacy is high (ERR >95%), but the difference between the two means is expected to increase as efficacy decreases. It is also worth noting that two out of the 13 studies with assessment of efficacy within 3 weeks would not meet the WHO threshold for acceptable efficacy of 90% [3] for ERR when calculated by AM, while only 8% would not meet the WHO threshold, if calculated by GM (2/24). Therefore, between the two, using AM, as suggested by WHO, appears to be more sensitive an approach to identify problems with the response to praziquantel.

We employed models to help further qualifying these findings using explanatory variables. We found that ERRs are more consistent between the AM and GM egg count values of S. haematobium and S. japonicum than of S. mansoni infection, and for school-aged children and adults than for preschool-aged children (but only when using GM). On the contrary, these findings are not accounted for by the baseline and post-treatment distributions of the raw egg counts (S2 Table), intensities of infection (S3 Table), or proportion of individuals with extreme values (S2 Table). Together, these findings are important to allow a meaningful comparison of newer studies using AM to older studies which would have used GM.

We then compared drug efficacy estimates by ERR (using AM or GM) versus CR. While generally used in the past, the CR is known to have some major limitations, and is no longer recommended by WHO for assessing the programmatic efficacy of drugs used in mass drug administration [12,43,45]. As expected, efficacy estimates by ERR and CR were hardly comparable, as they assess two different outcome measures (intensity versus presence of infection). At the onset of schistosomiasis control, the primary goal is to reduce morbidity, which is reflected by infection intensity, and hence, the ERR rather than CR might be the efficacy measure of choice. This choice is further justified by the relatively low sensitivity of widely used diagnostic methods, particularly the Kato-Katz technique for intestinal schistosomiasis [22,24,42] and the fact that current anthelmintic drugs only show low to moderate efficacies in terms of CR [13,46,47].

However, the situation is complex, and neither ERR nor CR alone provide a satisfactory description of the situation. The key questions about outcome measures are between effects on presence versus intensity of infection; and between measures of central tendency (for a group of individuals) versus individual subject responses. Our analyses indicate that the distribution of individual responses in egg excretion may be a better way of expressing results, as it comprises in one single measure drug effects on both presence and intensity of infection, and allows further detailing the distribution in centiles–which helps identifying and quantitating the presence of poor responders. ERRAM and iERR are correlated, but the latter appears to be more apt to detect the presence and to quantitate the magnitude of suboptimal responses to praziquantel. More than 10% of individuals had ERR <90% in five studies, only two of which were identified by applying the ERRAM 90% threshold. In these two studies, with ERRAM ~82%, <30% of individuals did not achieve a 90% reduction in their egg counts, and 11–17% had no reduction at all.

These observations raise important questions as to which approach is best suited to assess drug efficacy for which purpose: while ERR is, currently, the preferred measure at the program level, identifying poor responders is important in view of tracking trends in responses and signaling potential problem areas and emergence of drug resistance. Importantly, discussions should be held with a variety of stakeholders, including drug regulatory authorities, especially if new drug applications are forthcoming.

Lastly, we compared drug efficacy estimates (ERR calculated using AM and GM, and CR) obtained by a single versus quadruplicate Kato-Katz thick smears for S. mansoni. Obviously, a single Kato-Katz thick smear has a lower sensitivity than multiple Kato-Katz thick smears (here ~81%) [48], and the effect and ramifications thereof for estimating drug efficacy against Schistosoma and other helminth infections have been discussed [42,4951]. At the same time, collecting single stool samples and examining single Kato-Katz thick smears is operationally more feasible and less expensive than multiple sampling and multiple thick smear examinations under a microscope. We found that, when single Kato-Katz thick smears are used, the initial intensities of infection are ~2.5 times higher than with quadruplicate thick smears; efficacy estimation by ERRs based on AM and CR are ~8–9% higher, respectively, whereas ERRs based on GM are similar. Taken together, these results reflect the lower sensitivity of a single Kato-Katz thick smear, which misses low-intensity infections both on enrolment and post-treatment; the bias appears to be proportionally greater for the initial infection intensity than for the treatment outcomes. Overestimation of treatment effects may be an issue with efficacies nearing the 90% ERR threshold. In order to ascertain whether this introduces a selection bias which could affect the estimation of efficacy [52] (i.e., if the sample positive on a single Kato-Katz thick smear is different from that positive on multiple Kato-Katz thick smears), we compared both baseline FECs and efficacy estimates in these two samples using the same diagnostic technique; we found that excluding the ~19% of subjects who were negative on a single Kato-Katz selects for subjects with marginally higher initial FECs, but has ultimately no effect on efficacy estimates. Currently there is no diagnostic ‘gold’ standard method to assess Schistosoma response to treatment [53]. Standards may need to be tailored to the study, whether a field survey (limited by practical imperatives) or a clinical trial (which could afford more complex conditions and costly diagnosis), and whether in high or low infection intensity setting. Part of the problem, however, is the limited sensitivity of the current diagnostic methods, particularly for the detection of low intensity infections, which cannot be corrected until and unless more reliable tests become available [6,42,54].

Conclusions

Using group means is practical when assessing sample effects, but may not be suited to detect small changes, especially those that may occur in early phases of decreasing drug efficacy. We estimate that the distribution of individual responses in Schistosoma egg excretion, which accounts for individual variability of responses to praziquantel treatment, allows measuring effects on both presence and intensity of infection, and helps identifying and quantitating poor responders. More research and larger databases will be required in order to identify meaningful thresholds–e.g., centile by which 90% ERR is achieved; ERR achieved by the lowest 5% or 10% centile–and also analyze in greater detail reasons for poor response.

Both approaches could be used in parallel and complement each-other. It is important to agree on standardized outcome measures that are tailored to specific purposes, such as epidemiological surveys, routine monitoring, clinical trials, morbidity control, or elimination settings. Hence, we invite other groups to contribute to this discussion and scientific inquiry.

Supporting Information

S1 Table. Results of general linear models of distinct analyses of ERR based on geometric means and ERR based on arithmetic means.

https://doi.org/10.1371/journal.pntd.0003821.s001

(XLSX)

S2 Table. Central values of the distribution of counts of eggs per gram and log transformed counts of eggs per gram measured at baseline for all studies.

https://doi.org/10.1371/journal.pntd.0003821.s002

(XLSX)

S3 Table. Frequencies and proportions of species by intensities of infection at baseline.

https://doi.org/10.1371/journal.pntd.0003821.s003

(XLSX)

Acknowledgments

Thanks to Dr. Jing Xu from the National Institute of Parasitic Diseases, Chinese Center for Disease Control and Prevention, for kindly providing S. japonicum data. We also wish to thank Florilène Bouisset for statistical advice.

Disclaimer. The opinions expressed in this paper are those of the authors and may not reflect those of their employing organizations. PLO is a staff member of the World Health Organization (WHO) and MV is a staff member of LIH; the authors alone are responsible for the views expressed in this publication and it does not necessarily represent the decisions, policy, or views of their organizations.

Author Contributions

Conceived and designed the experiments: PLO MV AD. Performed the experiments: PLO MV AD. Analyzed the data: PLO MV AD. Contributed reagents/materials/analysis tools: MV. Wrote the paper: PLO MV AD JTC AG JK CHK SK AL EKN GR AUS JCSF KS XNZ JU.

References

  1. 1. Colley DG, Bustinduy AL, Secor WE, King CH (2014) Human schistosomiasis. Lancet.
  2. 2. WHO (2006) Preventive chemotherapy in human helminthiasis: coordinated use of anthelminthic drugs in control interventions: a manual for health professionals and programme managers. Geneva: World Health Organization. 1–62 p.
  3. 3. WHO (2013) Assessing the efficacy of anthelmintic drug against schistosomiasis and soil-transmitted helminthiases. Geneva: World Health Organization. 1–29 p.
  4. 4. WHO (2012) Schistosomiasis: population requiring preventive chemotherapy and number of people treated in 2010. Wkly Epidemiol Rec 87: 37–44. pmid:22308580
  5. 5. Schur N, Vounatsou P, Utzinger J (2012) Determining treatment needs at different spatial scales using geostatistical model-based risk estimates of schistosomiasis. PLoS Negl Trop Dis 6: e1773. pmid:23029570
  6. 6. Knopp S, Becker SL, Ingram KJ, Keiser J, Utzinger J (2013) Diagnosis and treatment of schistosomiasis in children in the era of intensified control. Expert Rev Anti Infect Ther 11: 1237–1258. pmid:24127662
  7. 7. Zhou XN, Xu J, Chen HG, Wang TP, Huang XB, et al. (2011) Tools to support policy decisions related to treatment strategies and surveillance of schistosomiasis japonica towards elimination. PLoS Negl Trop Dis 5: e1408. pmid:22206024
  8. 8. WHO (2002) Prevention and control of schistosomiasis and soil transmitted helminthiasis: report of a WHO expert committee. Geneva: World Health Organization Tech Rep Ser 912: 1–57.
  9. 9. van Lieshout L, Polderman AM, Deelder AM (2000) Immunodiagnosis of schistosomiasis by determination of the circulating antigens CAA and CCA, in particular in individuals with recent or light infections. Acta Trop 77: 69–80. pmid:10996122
  10. 10. Corstjens PL, CJ DED, Kornelis D, Tjon Kon Fat EM, Wilson RA, et al. (2014) Tools for diagnosis, monitoring and screening of Schistosoma infections utilizing lateral-flow based assays and upconverting phosphor labels. Parasitology: 1–15.
  11. 11. Stothard JR, Sousa-Figueiredo JC, Navaratnam AM (2013) Advocacy, policies and practicalities of preventive chemotherapy campaigns for African children with schistosomiasis. Expert Rev Anti Infect Ther 11: 733–752. pmid:23879611
  12. 12. Montresor A (2011) Cure rate is not a valid indicator for assessing drug efficacy and impact of preventive chemotherapy interventions against schistosomiasis and soil-transmitted helminthiasis. Trans Royal Soc Trop Med Hyg 105: 361–363. pmid:21612808
  13. 13. Keiser J, Utzinger J (2008) Efficacy of current drugs against soil-transmitted helminth infections: systematic review and meta-analysis. JAMA 299: 1937–1948. pmid:18430913
  14. 14. Gabrielli AF, Montresor A, Chitsulo L, Engels D, Savioli L (2011) Preventive chemotherapy in human helminthiasis: theoretical and operational aspects. Trans R Soc Trop Med Hyg 105: 683–693. pmid:22040463
  15. 15. Smothers CD, Sun F, Dayton AD (1999) Comparison of arithmetic and geometric means as measures of a central tendency in cattle nematode populations. Vet Parasitol 81: 211–224. pmid:10190865
  16. 16. Montresor A, Engels D, Chitsulo L, Gabrielli A, Albonico M, et al. (2011) The appropriate indicator should be used to assess treatment failure in STH infections. Am J Trop Med Hyg 85: 579–580; author reply 581. pmid:21896826
  17. 17. Dobson RJ, Sangster NC, Besier RB, Woodgate RG (2009) Geometric means provide a biased efficacy result when conducting a faecal egg count reduction test (FECRT). Vet Parasitol 161: 162–167. pmid:19135802
  18. 18. Levecke B, Brooker SJ, Knopp S, Steinmann P, Sousa-Figueiredo JC, et al. (2014) Effect of sampling and diagnostic effort on the assessment of schistosomiasis and soil-transmitted helminthiasis and drug efficacy: a meta-analysis of six drug efficacy trials and one epidemiological survey. Parasitology in press.
  19. 19. Yu JM, de Vlas SJ, Yuan HC, Gryseels B (1998) Variations in fecal Schistosoma japonicum egg counts. Am J Trop Med Hyg 59: 370–375. pmid:9749627
  20. 20. WHO (1991) Basic laboratory methods in medical parasitology. Geneva: World Health Organization. 1–61 p.
  21. 21. Lin DD, Liu JX, Liu YM, Hu F, Zhang YY, et al. (2008) Routine Kato-Katz technique underestimates the prevalence of Schistosoma japonicum: a case study in an endemic area of the People's Republic of China. Parasitol Int 57: 281–286. pmid:18485807
  22. 22. Engels D, Sinzinkayo E, de Vlas SJ, Gryseels B (1997) Intraspecimen fecal egg count variation in Schistosoma mansoni infection. Am J Trop Med Hyg 57: 571–577. pmid:9392598
  23. 23. Utzinger J, Booth M, N'Goran EK, Müller I, Tanner M, et al. (2001) Relative contribution of day-to-day and intra-specimen variation in faecal egg counts of Schistosoma mansoni before and after treatment with praziquantel. Parasitology 122: 537–544. pmid:11393827
  24. 24. de Vlas SJ, Gryseels B (1992) Underestimation of Schistosoma mansoni prevalences. Parasitol Today 8: 274–277. pmid:15463638
  25. 25. Stete K, Krauth SJ, Coulibaly JT, Knopp S, Hattendorf J, et al. (2012) Dynamics of Schistosoma haematobium egg output and associated infection parameters following treatment with praziquantel in school-aged children. Parasit Vectors 5: 298. pmid:23259435
  26. 26. Scherrer AU, Sjöberg MK, Allangba A, Traoré M, Lohourignon LK, et al. (2009) Sequential analysis of helminth egg output in human stool samples following albendazole and praziquantel administration. Acta Trop 109: 226–231. pmid:19070583
  27. 27. Garba A, Lamine MS, Barkire N, Djibo A, Sofo B, et al. (2013) Efficacy and safety of two closely spaced doses of praziquantel against Schistosoma haematobium and S. mansoni and re-infection patterns in school-aged children in Niger. Acta Trop 128: 334–344. pmid:22940014
  28. 28. Coulibaly JT, Fürst T, Silué KD, Knopp S, Hauri D, et al. (2012) Intestinal parasitic infections in schoolchildren in different settings of Côte d'Ivoire: effect of diagnostic approach and implications for control. Parasit Vectors 5: 135. pmid:22768986
  29. 29. Landouré A, Dembélé R, Goita S, Kané M, Tuinsma M, et al. (2012) Significantly reduced intensity of infection but persistent prevalence of schistosomiasis in a highly endemic region in Mali after repeated treatment. PLoS Negl Trop Dis 6: e1774. pmid:22860153
  30. 30. Sousa-Figueiredo JC, Betson M, Atuhaire A, Arinaitwe M, Navaratnam AM, et al. (2012) Performance and safety of praziquantel for treatment of intestinal schistosomiasis in infants and preschool children. PLoS Negl Trop Dis 6: e1864. pmid:23094120
  31. 31. Olliaro PL, Vaillant MT, Belizario VJ, Lwambo NJ, Ouldabdallahi M, et al. (2011) A multicentre randomized controlled trial of the efficacy and safety of single-dose praziquantel at 40 mg/kg vs. 60 mg/kg for treating intestinal schistosomiasis in the Philippines, Mauritania, Tanzania and Brazil. PLoS Negl Trop Dis 5: e1165. pmid:21695161
  32. 32. Xu J, Guo JG, Wu XH, Zeng XJ, Yang WP, et al. (2009) Efficacy and adverse effects of film coated praziquantel for treatment of schistosomiasis japonica. Zhonghua Yu Fang Yi Xue Za Zhi 43: 718–722 (in Chinese). pmid:20021854
  33. 33. Raso G, N'Goran EK, Toty A, Luginbühl A, Adjoua CA, et al. (2004) Efficacy and side effects of praziquantel against Schistosoma mansoni in a community of western Côte d'Ivoire. Trans R Soc Trop Med Hyg 98: 18–27. pmid:14702835
  34. 34. N'Goran EK, Gnaka HN, Tanner M, Utzinger J (2003) Efficacy and side-effects of two praziquantel treatments against Schistosoma haematobium infection, among schoolchildren from Côte d'Ivoire. Ann Trop Med Parasitol 97: 37–51. pmid:12662421
  35. 35. Utzinger J, N'Goran EK, N'Dri A, Lengeler C, Tanner M(2000) Efficacy of praziquantel against Schistosoma mansoni with particular consideration for intensity of infection. Trop Med Int Health 5: 771–778. pmid:11123824
  36. 36. Olds GR, King C, Hewlett J, Olveda R, Wu G, et al. (1999) Double-blind placebo-controlled study of concurrent administration of albendazole and praziquantel in schoolchildren with schistosomiasis and geohelminths. J Infect Dis 179: 996–1003. pmid:10068597
  37. 37. Utzinger J, N'Goran EK, Esse Aya CM, Acka Adjoua C, Lohourignon KL, et al. (1998) Schistosoma mansoni, intestinal parasites and perceived morbidity indicators in schoolchildren in a rural endemic area of western Côte d'Ivoire. Trop Med Int Health 3: 711–720. pmid:9754666
  38. 38. Albonico M, Bickle Q, Haji HJ, Ramsan M, Khatib KJ, et al. (2002) Evaluation of the efficacy of pyrantel-oxantel for the treatment of soil-transmitted nematode infections. Trans R Soc Trop Med Hyg 96: 685–690. pmid:12625151
  39. 39. Rodríguez de Gil P, Thanh Pham JR, Nguyen D, Kromrey JD, Kim ES (2013) SAS Macros CORR_P and TANGO: Interval Estimation for the Difference Between Correlated Proportions in Dependent Samples. SESUG 2013: SAS Institute. pp. 1–9.
  40. 40. Tang ML, Li HQ, Tang NS (2012) Confidence interval construction for proportion ratio in paired studies based on hybrid method. Stat Methods Med Res 21: 361–378. pmid:20876164
  41. 41. Zou H, Hastie T (2005) Regularization and Variable Selection via the Elastic Net. Journal of the Royal Statistical Society, Series B 67: 301–320.
  42. 42. Bergquist R, Johansen MV, Utzinger J (2009) Diagnostic dilemmas in helminthology: what tools to use and when? Trends Parasitol 25: 151–156. pmid:19269899
  43. 43. Vercruysse J, Behnke JM, Albonico M, Ame SM, Angebault C, et al. (2011) Assessment of the anthelmintic efficacy of albendazole in school children in seven countries where soil-transmitted helminths are endemic. PLoS Negl Trop Dis 5: e948. pmid:21468309
  44. 44. Montresor A (2007) Arithmetic or geometric means of eggs per gram are not appropriate indicators to estimate the impact of control measures in helminth infections. Trans R Soc Trop Med Hyg 101: 773–776. pmid:17544470
  45. 45. Vercruysse J, Albonico M, Behnke JM, Kotze AC, Prichard RK, et al. (2011) Is anthelmintic resistance a concern for the control of human soil-transmitted helminths? International Journal for Parasitology: Drugs and Drug Resistance 1: 14–27. pmid:24533260
  46. 46. Danso-Appiah A, Utzinger J, Liu J, Olliaro P (2008) Drugs for treating urinary schistosomiasis. Cochrane Database Syst Rev: Cd000053.
  47. 47. Danso-Appiah A, Olliaro PL, Donegan S, Sinclair D, Utzinger J (2013) Drugs for treating Schistosoma mansoni infection. Cochrane Database Syst Rev 2: Cd000528. pmid:23450530
  48. 48. McCarthy JS, Lustigman S, Yang GJ, Barakat RM, Garcia HH, et al. (2012) A research agenda for helminth diseases of humans: diagnostics for control and elimination programmes. PLoS Negl Trop Dis 6: e1601. pmid:22545166
  49. 49. Utzinger J, N'Goran EK, N'Dri A, Lengeler C, Xiao S, et al. (2000) Oral artemether for prevention of Schistosoma mansoni infection: randomised controlled trial. Lancet 355: 1320–1325. pmid:10776745
  50. 50. Knopp S, Mgeni AF, Khamis IS, Steinmann P, Stothard JR, et al. (2008) Diagnosis of soil-transmitted helminths in the era of preventive chemotherapy: effect of multiple stool sampling and use of different diagnostic techniques. PLoS Negl Trop Dis 2: e331. pmid:18982057
  51. 51. Knopp S, Speich B, Hattendorf J, Rinaldi L, Mohammed KA, et al. (2011) Diagnostic accuracy of Kato-Katz and FLOTAC for assessing anthelmintic drug efficacy. PLoS Negl Trop Dis 5: e1036. pmid:21532740
  52. 52. Xu J, Feng T, Guo J, Zheng H, Wang Q, et al. (2005) Comprehensive evaluation of several diagnosis agents of schistosomiasis japonica in China Chin J Schisto Contr 17: 116–119 (in Chinese).
  53. 53. Wang XH, Zhou XN, Vounatsou P, Chen Z, Utzinger J, et al. (2008) Bayesian spatio-temporal modeling of Schistosoma japonicum prevalence data in the absence of a diagnostic 'gold' standard. PLoS Negl Trop Dis 2: e250. pmid:18545696
  54. 54. Utzinger J, N'Goran E K, Caffrey CR, Keiser J (2011) From innovation to application: social-ecological context, diagnostics, drugs and integrated control of schistosomiasis. Acta Trop 120 Suppl 1: S121–137. pmid:20831855