Background and Objectives
In Brazil, as in many other affected countries, a large proportion of visceral leishmaniasis (VL) occurs in remote locations and treatment is often performed on basis of clinical suspicion. This study aimed at developing predictive models to help with the clinical management of VL in patients with suggestive clinical of disease.
Cases of VL (n = 213) had the diagnosis confirmed by parasitological method, non-cases (n = 119) presented suggestive clinical presentation of VL but a negative parasitological diagnosis and a firm diagnosis of another disease. The original data set was divided into two samples for generation and validation of the prediction models. Prediction models based on clinical signs and symptoms, results of laboratory exams and results of five different serological tests, were developed by means of logistic regression and classification and regression trees (CART). From these models, clinical-laboratory and diagnostic prediction scores were generated. The area under the receiver operator characteristic curve, sensitivity, specificity, and positive predictive value were used to evaluate the models' performance.
Based on the variables splenomegaly, presence of cough and leukopenia and on the results of five serological tests it was possible to generate six predictive models using logistic regression, showing sensitivity ranging from 90.1 to 99.0% and specificity ranging from 53.0 to 97.2%. Based on the variables splenomegaly, leukopenia, cough, age and weight loss and on the results of five serological tests six predictive models were generated using CART with sensitivity ranging from 90.1 to 97.2% and specificity ranging from 68.4 to 97.4%. The models composed of clinical-laboratory variables and the rk39 rapid test showed the best performance.
The predictive models showed to be a potential useful tool to assist healthcare systems and control programs in their strategical choices, contributing to more efficient and more rational allocation of healthcare resources.
Visceral leishmaniasis (VL) is a neglected tropical disease endemic to 65 countries, including Brazil, where the disease frequently occurs in remote locations and treatment is often performed on the basis of clinical suspicion. Predictive models based on scoring systems could be a helpful tool for the clinical management of VL. Based on clinical signs and symptoms, and five different serological tests of 213 patients with parasitologically confirmed (cases) and 119 with clinical suspicion of VL but with another confirmed etiology (non-cases), twelve prediction models using logistic regression and classification and regression trees (CART) for VL diagnosis were developed. The model composed of the clinical-laboratory variables and the rk39 rapid test showed the best performance in both logistic regression and CART (Sensitivity of 90.1% and specificity ranging from 97.2–97.4%). The scoring system is simple and based on the clinical-laboratory findings that are easily available in most clinical settings. The results suggest that those models might be useful in locations where access to available diagnostic methods is difficult, contributing to more efficient and more rational allocation of healthcare resources.
Citation: Machado de Assis TS, Rabello A, Werneck GL (2012) Predictive Models for the Diagnostic of Human Visceral Leishmaniasis in Brazil. PLoS Negl Trop Dis 6(2): e1542. doi:10.1371/journal.pntd.0001542
Editor: Richard Reithinger, George Washington University, United States of America
Received: August 16, 2011; Accepted: January 10, 2012; Published: February 28, 2012
Copyright: © 2012 Machado de Assis et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by the Oswaldo Cruz Foundation (FIOCRUZ) and National Counsel of Technological and Scientific Development (CNPq). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Visceral leishmaniasis (VL) is a neglected tropical disease caused by the intracellular protozoan parasite Leishmania infantum (syn. Leishmania chagasi). The disease is endemic to 65 countries and 90% of world cases are reported in India, Bangladesh, Nepal, Ethiopia, Brazil, and the Sudan . In Brazil, more then 15.000 VL cases were reported between 2007 and 2010, with 880 deaths . The disease primarily affects the poorest people and is fatal if untreated. The control strategies used in Brazil to reduce the disease morbidity and mortality rates consists on the early diagnosis and treatment of human cases and the control of the populations of domestic reservoirs and vectors .
Early diagnosis is a challenge in Brazil, as in other affected countries, where the disease is still frequently treated only on the basis of clinical suspicion. Clinically, the disease is characterized by prolonged fever, substantial weight loss, hepatomegaly, splenomegaly, pancytopenia, hypergammaglobulinemia , . The firm diagnosis of VL needs to rely on efficacious laboratorial support. The current reference test for disease diagnostic is the microscopic demonstration of Leishmania in spleen, bone marrow, lymph nodes or liver aspirates, but both the aspiration procedure and the reading of slides require a high level of expertise that makes them unsuitable for generalized field use , .
Several serological diagnostic methods have been widely evaluated for the diagnosis of VL, such as the enzyme linked immunosorbent assay (ELISA) with different antigens and the indirect fluorescence antibody test (IFAT). In Brazil, IFAT is the serologic test made available by the Public Health System. ELISAs and IFAT depend on equipment and laboratorial infrastructure. Two other tests easy to use have been appointed as appropriate for the diagnosis of VL in control programs: the Direct Agglutination Test (DAT) and the rK39 rapid tests –.
The development of predictive models could help in the management of patients, especially in towns where the access to diagnostic methods is difficult, being useful as a cost-effective tool in a health care system with limited resources. This study aimed at developing models based on scoring systems using logistic regression and classification and regression trees (CART) to predict the occurrence of VL in patients with suggestive clinical of disease in Brazil.
Settings and patient selection
The models were developed using a database generated from a prospective study conducted in four states of Brasil, published elsewhere , . We evaluated a group of 332 patients with symptoms and/or signs suggestive of VL referred for diagnostic and eventual treatment in states of Maranhão (Federal University of Maranhão, 35 patients enrolled), Piauí (Federal University of Piauí, 121 patients), Bahia (Gonçalo Muniz Research Center, 119 patients) and Minas Gerais (René Rachou Research Center, 57 patients), from May 2005 and May 2007.
By the end of clinical investigation, all VL cases had the diagnosis confirmed by parasitological methods. The non-cases had suggestive clinical presentation of VL, a negative parasitological diagnosis and the accomplished diagnosis of another disease. The non-cases were diagnosed with various diseases, such as leukemia, liver disease, schistosomiasis, ascariasis, liver fibrosis, lymphoma, rheumatoid arthritis, malaria, mononucleosis, typhoid fever, marrow aplasia, liver cirrhosis, meningitis, lupus erythematosus, encephalitis, tuberculosis, among others.
Procedures and diagnostic tests
Patients underwent a standardized interview regarding epidemiological and clinical history and a physical examination. IFAT was performed with an industrial kit (Biomanguinhos, Rio de Janeiro, Brazil) according to the manufacturer's instructions. Samples scored positive when fluorescent microscopy showed clear evidence that they produced a cytoplasmic or membranous fluorescence with promastigotes using a cut-off dilution of 1:80. L. chagasi-ELISA and rK39-ELISA were performed according to Assis et al. (2008) . The cutoff of reactions was determined as the mean plus two standard deviations of the absorbance of control sera (n = 20). DAT was performed according to Pedras et al. (2008) . The cutoff value was determined by analyzing the receiver operator characteristic curve. Rapid test (IT-LEISH® Diamed Latino-America S. A. - Cressier sur Morat, Switzerland) was performed according to the manufacturer's instructions and Assis et al. (2008) . The test was positive when two red lines appeared in the middle of the nitrocellulose membrane, negative when only one redline appeared and invalid when no line was evident. The rapid test and the bone marrow aspirate were performed at the center of origin of the patients evaluated; all other serological tests were performed at the Rene Rachou Research Center.
The Research Ethics Committee of René Rachou Research Center and all other institutions involved in this study had previously approved the informed consent forms and procedures. Written informed consent was obtained from all the adults and from minors' parents or legal guardians. The study was conducted in agreement with the principles of the Helsinki Declaration and the Resolution 196/96 of the National Health Council of the Ministry of Health that regulates research involving human subjects in Brazil (CEPSH/CPqRRn°: 13/2003).
The original data set was randomly divided into 2 parts: the “test sample” (patients from Maranhão, Piauí and Minas Gerais, n = 213) was used to construct the models and the “validation sample” (patients from Bahia, n = 119) was used to validate the models. Predictive models were built using logistic regression and CART. Statistical analyses were performed using Stata, version 10.0 (Stata), and Splus, version 4.5 (StatSci).
For developing predictive models with logistic regression, initially the most important factors associated with the occurrence of visceral leishmaniasis were identified. A p-value of ≤0.2 for the univariate association with visceral leishmaniasis was used for selecting variables for the multivariate model. A stepwise elimination procedure was performed, using a p-value de ≤0.05 as the criterion for variables to remain in the model.
A predictive model based on a scoring system, with points allocated to each prognostic factor, was created from the final logistic regression model run in the test sample. The scoring system was generated by dividing the value of the regression coefficient of each variable by the smallest coefficient and rounding the quotients to the closest integer . Posteriorly, the final score was obtained through the sum of points attributed to the presence of each predictive variable that remained in the final model and to the results of five diagnostic methods: IFAT, L. chagasi-ELISA, rK39-ELISA, DAT and rapid test.
For constructing predictive model using CART all available variables were initially included in the analysis. The CART method was used to build a binary classification tree through successive partitions, dividing the data into more homogeneous subgroups at each split (“node”). At each node, the algorithm selected the variable with the greatest capacity for discriminating between the 2 outcome groups (VL and non-VL). The first division of the tree corresponds to the variable with the greatest ability to discriminate between VL cases and non-VL patients; the discriminatory power decreases with each subsequent division (“branch”).
The CART algorithm adds nodes until they are homogenous or contains few observations. The problem of creating a useful tree is to find suitable guidelines to achieve a tree with a lower level of misclassification but, at the same time, not too much adjusted to the data. This can be accomplished by downsizing (“pruning”) the tree. The general principle of pruning is that the tree of best size would have the lowest misclassification rate for an individual not included in the original data . Pruning was achieved by decreasing the number of nodes without a significant increase of deviance, with the aid of a graph that shows the relationship between deviance and the number of nodes on the tree . The best tree suggested by our analysis had 7 leaves.
The sensitivity, specificity, positive predictive value (PPV) and area under the receiver operator characteristic (ROC) curve were used to evaluate the performance of the models. The sensitivity is the probability of the test result be positive among patients with the disease, specificity is the probability of the test result be negative among patients without the disease and PPV is the probability that a patient has the disease given a positive test result. The ROC curve consists of a graph of sensitivity versus false positive rate and the area under this curve provides a summary of the ability of a test to discriminate two groups (here, VL and non-VL patients).
Three hundred thirty-two patients were included in the analysis, 213 parasitologically confirmed VL cases and 119 non-cases with clinical suspicion of VL but with another confirmed etiology. Detailed description of the group and validation of the rK39 rapid test and DAT is reported by Assis et al. (2011) . The average age of the VL cases in test sample was 21 years (1 month to 74 years), and 63% (n = 88) were female and the average age of the non-cases was 16 years (2 months to 66 years), and 60% (n = 44) were female. Table 1 shows the clinical and laboratory characteristics of subjects in the test sample. Table 2 shows the predictive variables that remained in the final logistic regression model: Splenomegaly, leukopenia and cough. The score system generated by using logistic regression attributed −1 point for cough, 1 point for leukopenia, 3 points for splenomegaly and positive IFAT, 4 points for positive L. chagasi ELISA, 5 points for positive rK39 rapid test, 6 points for positive rK39-ELISA and 7 points for positive DAT (Table 3).
The CART model was composed by the variables splenomegaly, leukopenia, cough, age and weight loss (Figure 1). The variable with the greatest discriminative power was splenomegaly. The probabilities of VL, as predicted in the leaves of the tree, ranged from 0% to 87%.
Classification and regression tree model for predicting VL. The number of patients (N) and the probability of VL (% with VL) are shown at each node. Terminal nodes are shaded.
Table 3 and 4 show the comparison of the predictive performance of different models generated using logistic regression and CART in terms of the area under the ROC curve, sensitivity, specificity, and PPV evaluated in both test and validation samples. Using logistic regression it was possible to generate predictive models for the diagnostic of VL with sensitivity ranging from 90.1 to 99.0% and specificity ranging from 53.0 to 97.2%. Using CART it was possible to generate predictive models for VL with sensitivity ranging from 90.1 to 97.2% and specificity ranging from 68.4 to 97.4%. Logistic regression and CART in the test sample and validation sample had similar performance for most models.
Figure 2 presents one example, based in models developed using logistic regression, on how a chart could be used to help health professionals with the tests interpretation and the physicians with the clinical decision. In the validation sample, in the first model, composed only by clinical-laboratory variables, patients with score ≥3 (82/107–51%) showed a probability of having VL of 79% (data not shown). In the second model, when IFAT was added, patients with score ≥4 (78/107–73%) had a probability of VL of 90%. In the third model, which included clinical and laboratorial features and L. chagasi ELISA, patients with a score ≥5 (68/107–63%) had 94.1% probability of VL. In the model combining clinical-laboratory variables plus rK39 ELISA, patients with score ≥7 (72/107–67%) presented 94.4% VL probability. In the fifth model, adding DAT to clinical and laboratorial findings, patients with score ≥5 (68/107–63%) showed also a 94% probability of a VL diagnostic. Still, in the sixth model (clinical-laboratory plus rK39 rapid test) patients with score ≥5 (65/107–61%) showed the higher VL probability (PPV 98.5%).
VL is a serious disease, with repeatedly recognition of the lack of sufficient means for its elimination. Rapid diagnostic and adequate treatment of cases would certainly help to reduce morbidity and mortality and it may contribute also to decrease transmission where anthroponotic VL transmission occurs. Clinical diagnosis of VL is inaccurate because it's clinical presentation shares common features to several other diseases and can vary in different endemic areas. In the present study, splenomegaly, leukopenia and cough were the clinical-laboratory variables that remained in the predictive model using logistic regression; and splenomegaly, leukopenia, cough, age and weight loss were the clinical-laboratory variables that remained in the predictive model using CART for VL diagnosis.
Splenomegaly is a classic sign of VL that with the advance of disease can cause abdominal distension and pain. In the study by Tanoli et al. (2005)  in Pakistan, 95% of patients had splenomegaly and in the study by Daher et al. (2008)  and Rocha et al. (2011)  in Brazil, 96% and 94% of the patients, respectively, showed this signal, as well. Leukopenia and weigh loss are reported frequently in clinical studies involving patients with VL. In the study by Dursun et al. (2009)  in Turkey, 74% of the patients had leukopenia, and in the study by Queiroz et al. (2004)  and Daher et al. (2008)  in Brazil, 85% of the patients showed leukopenia and 95% showed weigh loss, respectively. Several authors have reported that the VL is predominant in children early in life and is associated with high morbidity and high number of deaths –. Other manifestations can be seen less consistent with the LV, such as cough and diarrhea , . In this study cough was a sign negatively correlated to LV. Therefore, VL should be suspected in endemic areas when patients present enlarged spleen, leukopenia and weigh loss, especially in children early in life.
Laboratory diagnosis of VL is, still now, complex. The sensitivity of parasitological tests is suboptimal, ranging from 53–86% for bone marrow up to 93–99% for spleen aspirates . Diagnostic research in VL has been damaged by the lack of a perfect gold standard. An alternative to the classical validation approach using parasitological diagnostic methods as the gold standard is the latent class analysis (LCA). LCA is based on the concept that the observed results of different imperfect tests for the same disease are influenced by a latent common variable, the true disease status, which cannot be directly measured –. Several studies used LCA methodology for the evaluation of diagnostic tests for VL, such as Boelaert et al. (1999, 2004 and 2008) –, Horst et al. (2009)  and Menten et al. (2008) .
Less invasive methods are being evaluated for VL diagnosis. IFAT, ELISA, and a polymerase chain reaction are examples of these efforts. Unfortunately, all of these tests require laboratory infrastructure and specialized professionals. More recently, alternatives to the methods mentioned above, such as DAT and rapid test have become available. DAT and rK39 show high sensitivity, specificity, rapid results and are easy to use –. In the multicenter study performed in Brazil, which served as the basis for the development of the predictive models presented, the IFAT showed sensitivity of 88% and specificity of 81%, the L. chagasi ELISA showed sensitivity of 92% and specificity of 77%, the rK39-ELISA showed sensitivity of 97% and specificity of 84% , the rapid test IT-LEISH® showed sensitivity of 93% and specificity of 97% and the DAT showed sensitivity of 90% and a specificity of 96% .
In the present study, it was possible to generate predictive models for VL with good general predictive performance. It was observed that the generated models showed better performance compared to the model based only on clinical-laboratory variables, reinforcing the importance of diagnostic tests in patients' management. From the standpoint of performance and practicality, the sixth model, composed of clinical-laboratory variables and the rK39 rapid test, developed using both logistic regression and CART, may represent the best suitability for use in peripheral services and referral centers, since the rapid test is easy to perform and to interpret, with result available within 20 minutes. Other models, such as the second, composed of clinical-laboratory variables and the IFAT could be useful in services that have this technique already implemented.
Clinical prediction models have been developed to help physicians improve the assessment of an individual's risk of a disease or to predict an outcome, for a great number of diseases, such as tuberculosis and pneumonia. It is the first time that this type of predictive model is developed for human VL and it represents an innovative approach in disease diagnosis. It was out of the scope of this study to evaluate the interference of epidemics or the seasonality of the disease and the possible use of other models as the early warning systems (EWS) based on environmental variables that have been developed to predict the occurrence of epidemics of cutaneous leishmaniasis and could be also applied to VL .
The use of a control group (non-VL patients) with a variety of diseases that can mimic VL and representative of the population that seeks references centers for VL in Brazil is one of the strong features of this study, providing a realistic scenario for the use of the predictive models generated. However, there are also some methodological limitations in our study that should be considered before deciding to apply the results of models in clinical practice. First, although our modeling strategy used geographically different samples for deriving and validating the models, one need to be cautious about the possibility that the patients enrolled in our study may not be representative of populations from other settings. Second, the patients were already identified at admission to be at risk for VL, in this sense our models were developed for a population attending to referral centers and might not be useful in different circumstances. Third, the small size of the validation sample, as compared to sample derivation contributed to the relatively low precision of sensitivity, specificity, PPV and the area under the receiver operator characteristic estimates in the validated models. Fourth, the use of leukopenia as a predictor might impair the use of such models in many endemic areas where a complete blood count is difficult to be performed. Unfortunately, a model without leucopenia did not performed well in our sample. Therefore, the development of simpler models with good predictive performance in settings where blood counts are not readily available is a challenge that should be explored in other studies.
The scoring system derived from logistic regression and the classification scheme based on CART models are simple and based on the clinical-laboratory findings that are easily available in most clinical settings. The model composed of clinical-laboratory variables and the rK39 rapid test developed using both logistic regression and the model CART showed the best performance and it could be used in health services. This assessment tool could support a physician's decision but should not preclude his assistance.
We thank Alexandre S. C. Braga, Mariana J. Pedras, Edward Oliveira, Aldina Barral, Isadora C. de Siqueira, Carlos H. N. Costa, Dorcas L. Costa, Thiago A. Holanda, Victor Y. R. Soares, Mauro Biá, Arlene J. M. Caldas and Gustavo A. S. Romero, involved in the previous prospective study of validation of diagnostic tests for visceral leishmaniasis.
Conceived and designed the experiments: TSMdA AR GLW. Performed the experiments: TSMdA AR GLW. Analyzed the data: TSMdA AR GLW. Contributed reagents/materials/analysis tools: TSMdA AR GLW. Wrote the paper: TSMdA AR GLW.
- 1. World Health Organization (2010) Control of the Leishmaniases. Report of a meeting of the WHO Expert Committee on the Control of Leishmaniases. Geneva: World Health Organization.
- 2. Brasil. Ministério da Saúde. Sistema de Informação de Agravos de Notificação. Available: http://dtr2004.saude.gov.br/sinanweb/tabnet/dh?sinannet/leishvi/bases/leishvbrnet.def. Accessed 2011 Aug 08.
- 3. Departamento de Vigilância Epidemiológica, Secretaria de Vigilância em Saúde, Ministério da Saúde (2006) Manual de vigilância e controle da leishmaniose visceral. Brasília: Ministério da Saúde.
- 4. Zijlstra EE, El-Hassan AM (2001) Leishmaniasis in Sudan. Trans R Soc Trop Med Hyg 95: 27–58.
- 5. Ritmeijer K, Melaku Y, Mueller M, Kipngetich S, O'Keeffe C, et al. (2006) Evaluation of the new recombinant K39 rapid diagnostic test for Sudanese visceral leishmaniasis. Am J Trop Med Hyg 74: 76–80.
- 6. Sundar S, Singh RK, Bimal SK, Gidwani K, Mishra A, et al. (2007) Comparative evaluation of parasitology and serological tests in the diagnosis of visceral leishmaniasis in India: a phase III diagnostic accuracy study. Trop Med Int Health 12: 284–289.
- 7. Assis TSMde, Braga ASC, Pedras MJ, Oliveira E, Barral A, et al. (2011) Multi-centric prospective evaluation of rK39 rapid test and direct agglutination test for the diagnosis of visceral leishmaniasis in Brazil. Trans R Soc Trop Med Hyg 105: 81–85.
- 8. Bern C, Jha SN, Joshi AB, Thakur GD, Bista MB (2000) Use of the recombinant rK39 dipstick test and the direct agglutination test in a setting endemic for visceral leishmaniasis in Nepal. Am J Trop Med Hyg 63: 153–157.
- 9. Assis TSMde, Braga ASC, Pedras MJ, Barral A, Siqueira IC, et al. (2008) Validação do teste imunocromatográfico rápido IT-LEISH® para o diagnóstico da leishmaniose visceral humana. Epidemiol Serv Saúde 17: 105–116.
- 10. Pedras MJ, Viana LG, de Oliveira EJ, Rabello A (2008) Comparative evaluation of direct agglutination test, rK39 and soluble antigen ELISA and IFAT for the diagnosis of visceral leishmaniasis in Brazil. Trans R Soc Trop Med Hyg 102: 172–178.
- 11. Barquet N, Domingo P, Cayla JA, González J, Rodrigo C, et al. (1997) Prognostic factors in meningococcal disease. Development of a bedside predictive model and scoring system. JAMA 278: 491–496.
- 12. Clark LA, Pregibon D (1993) Tree-based models. In: Chambers JM, Hastie TJ, editors. Statistical Models in S. Chapman & Hall, New York. pp. 377–419.
- 13. Venables WN, Ripley BD (1999) Modern Applied Statistics with S-PLUS. Third Edition. New York: Springer.
- 14. Tanoli ZM, Rai ME, Gandabur AS (2005) Clinical presentation and management of visceral leishmaniasis. J Ayub Med Coll Abbottabad 17: 51–53.
- 15. Daher EF, Evangelista LF, Silva Júnior GB, Lima RSA, Aragão EB, et al. (2008) Clinical Presentation and Renal Evaluation of Human Visceral Leishmaniasis (Kala-azar): A Retrospective Study of 57 Patients in Brazil. Braz J Infect Dis 12: 329–332.
- 16. Rocha NA, Silva Junior GB, Oliveira MJC, Abreu KLS, Franco LSLG, et al. (2011) Visceral leishmaniasis in children: a cohort of 120 patients in a metropolitan city of Brazil. Tur J Pediatr 53: 154–160.
- 17. Dursun O, Erisir S, Yesilipek A (2009) Visceral childhood leishmaniasis in southern Turkey: experience of twenty years. Tur J Pediatr 51: 1–5.
- 18. Queiroz MJA, Alves JGB, Correia JB (2004) Visceral leishmaniasis: clinical and epidemiological features of children in an endemic area. J Pediatr 80: 141–146.
- 19. Hui SL, Walter SD (1980) Estimating error rates of diagnostic tests. Biometrics 36: 167–171.
- 20. Rindskopf D, Rindskopf W (1986) The value of latent class analysis in medical diagnosis. Stat Med 5: 21–27.
- 21. Randolph AG, Guyatt GH, Calvin JE, Doig G, Richardson WS (1998) Understanding articles describing prediction tools. Crit Care Med 26: 1603–1612.
- 22. Boelaert M, Sayda ES, Goetghebeur E, Gomes-Pereira S, Ray DL, et al. (1999) Latent class analysis permits unbiased estimates of the validity of DAT for the diagnosis of visceral leishmaniasis. Trop Med Int Health 4: 395–401.
- 23. Boelaert M, Rijal S, Regmi S, Singh R, Balmansingh K, et al. (2004) A comparative study of the effectiveness of diagnostic tests for visceral leishmaniasis. Am J Trop Hyg 70: 72–77.
- 24. Boelaert M, El-safi S, Hailu A, Mukhtar M, Rijal S, et al. (2008) Diagnostic tests for kala-azar: a multi-centre study of the freeze-dried DAT, rK39 strip test and KAtex in east Africa and the Indian subcontinent. Trans Roy Soc Trop Med Hyg 102: 32–40.
- 25. Horst R, Tefera T, Assefa G, Ebrahim AZ, Davidson RN, et al. (2009) Field evaluation of rK39 test and direct agglutination test for diagnosis of visceral leishmaniasis in a population with high prevalence of human immunodeficiency virus in Ethiopia. Am J Trop Med Hyg 80: 929–934.
- 26. Menten J, Boelaert M, Lesaffre E (2008) Bayesian latent class models with conditionally dependent diagnostic tests: a case study. Stat Med 27: 4469–4488.
- 27. Chaves LF, Pascual M (2007) Comparing models for early warning systems of negleted tropical diseases. PLoS Neg Trop Dis 1: 1–6.