Citation: Becker SL, Yap P, Horié NS, Alirol E, Barbé B, Bhatta NK, et al. (2016) Experiences and Lessons from a Multicountry NIDIAG Study on Persistent Digestive Disorders in the Tropics. PLoS Negl Trop Dis 10(11): e0004818. https://doi.org/10.1371/journal.pntd.0004818
Editor: Serap Aksoy, Yale School of Public Health, UNITED STATES
Published: November 3, 2016
Copyright: © 2016 Becker et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work is part of the NIDIAG European research network (Collaborative Project), supported by the European Union’s Seventh Framework Programme for research, technological development, and demonstration under grant agreement no. 260260. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Persistent digestive disorders can be defined as any diarrhea (i.e., three or more loose stools per day) lasting for at least two weeks and/or abdominal pain that persists for two weeks or longer [1–3]. These disorders cause considerable morbidity and human suffering, and hence, are reasons why people might seek primary health care. However, in resource-constrained settings of the tropics and subtropics, accurate point-of-care diagnostics are often lacking and treatment is empiric, particularly in remote rural areas with no laboratory infrastructure. As a result, the relative contribution of selected pathogens to the syndrome of persistent digestive disorders is poorly understood, and evidence-based guidelines for patient management in different social-ecological settings are scarce [4–6].
In order to improve the clinical management of patients with disorders caused by neglected tropical diseases (NTDs), the European Commission (EC) funded a five-year study—the Neglected Infectious diseases DIAGnosis (NIDIAG) research consortium. The overarching goal of the NIDIAG consortium is to develop and validate patient-centered diagnosis–treatment guidelines for use at the primary health care level in low- and middle-income countries (http://www.nidiag.org) [3,7–9]. Emphasis is placed on three syndromes: (i) persistent digestive disorders described here; (ii) persistent fever; and (iii) neurological disorders, the latter two of which are detailed in companion pieces published in the same issue of PLOS Neglected Tropical Diseases.
With regard to the study on persistent digestive disorders, the main aims are (i) to identify the most important NTDs and other infectious agents that give rise to this clinical syndrome, including their relative frequency; (ii) to assess and compare the accuracy of different diagnostic methods; and (iii) to determine clinical responses to commonly employed empiric treatment options for persistent digestive disorders . To this end, a case–control study has been implemented in four countries: Côte d’Ivoire and Mali in West Africa and Indonesia and Nepal in Asia. An integral part of the NIDIAG consortium is to ensure that good clinical practice (GCP) and good clinical laboratory practice (GCLP) are adhered to while conducting the studies [10,11]. A quality assurance system, which included the development and implementation of a set of standard operating procedures (SOPs), along with on-the-spot staff training and internal and external quality control activities, has been developed at the project level and introduced at each study site. The development of, and adherence to, SOPs within harmonized study protocols were considered crucial steps for maximizing the integrity of laboratory and clinical data across study settings. They also provided the basis on which quality control activities could be performed.
For Which Procedures Have SOPs Been Developed?
For the study on persistent digestive disorders, 33 specific SOPs have been developed (Supporting Information). As summarized in Table 1, detailed steps on clinical and laboratory procedures, data management, and quality assurance were described. With regard to clinical investigations, SOPs on history taking and clinical examination, assessing inclusion and exclusion criteria, patient recruitment, and study flow were developed (S1-S6). Detailed instructions on how to perform a set of laboratory diagnostic techniques for the detection of helminth and intestinal protozoa infections were included in the laboratory SOPs. Different conventional stool microscopy techniques were combined with more recent rapid antigen detection tests to encompass a broad spectrum of potentially implicated pathogens with high diagnostic accuracy (S7-S20). An overview of the employed diagnostic methods is provided in Table 2. Pertaining to data management, SOPs on completion of case report forms (CRFs) and on various activities (such as data entry, data cleaning, querying, database locking, and backing up data) were also included. To ensure quality control, SOPs on internal quality control activities, external monitoring, and laboratory supervision visits were jointly developed for the three syndromes (S21-S33).
Of note, all SOPs were developed in English (for use in Nepal) and subsequently translated into French (for use in Côte d’Ivoire and Mali) and Bahasa Indonesia (for use in Indonesia). This comprehensive set of closely interconnected SOPs—which provides guidance on all essential procedures from the first presentation of an individual at a health care center until the final processing of all patient and laboratory data—is displayed in Fig 1.
How Was the Development of SOPs Coordinated, and Which Quality Control Measures Were Adopted?
The development and harmonization of the various SOPs was coordinated by the quality assurance group of the NIDIAG consortium and the trial management group (TMG) of the digestive syndrome study and followed a standard template and consortium-wide guidelines stipulated in the SOP entitled “SOP on SOP” (S24). This allowed different authors with varied background and writing styles to convey key messages and pass on their expert knowledge in a systematic, standardized manner for the benefit of the end user of all the SOPs. In addition, it provided clear instructions on how the SOPs should be numbered, reviewed, and approved to allow for strict version control. The authors of the SOPs were chosen from within the NIDIAG consortium, and allocation of topics was based on expertise and track record in the clinical, laboratory, data management, and quality assurance components of the study. Experts in the field, at the bench, and at the bedside carefully reviewed and revised the draft SOPs. Before the start of recruitment, local clinical and laboratory teams were trained on the set of SOPs through two hands-on workshops lasting three days each that were conducted on site by relevant experts of the NIDIAG consortium. During these workshops, feedback from the local partners was incorporated to refine the already developed SOPs, and additional SOPs were jointly developed to meet specific demands of local clinical, epidemiologic, and laboratory conditions. For example, in Indonesia, where Kinyoun staining was not available, an SOP pertaining to a slightly modified acid-fast staining technique was developed for the local team instead. Finally, once an SOP was finalized, a member of the TMG would approve it. A quality assurance member of the NIDIAG consortium was tasked to compile and keep updated the final set of SOPs and ensure that the latest versions were available on the NIDIAG intranet for distribution among the different country partners.
Which Patient Recruitment Patterns Have Been Observed?
In clinical trials and case–control studies, it is one of the most difficult tasks to precisely define patient inclusion and exclusion criteria and to carry out the enrolment accordingly under “real life” conditions, thereby minimizing selection bias . Apart from a specific study protocol and a given research question, several studies have shown that many internal and external factors may considerably influence the pace of patient recruitment in clinical trials. Many of these factors are not related to scientific issues, such as the workload for study staff who are responsible for the recruitment of patients, the distance between the study center and the place of residence of potential participants, insufficient engagement with the community, and the complexity of consent procedures prior to the start of patient enrollment [13,14]. In the NIDIAG study pertaining to persistent digestive disorders, several site assessments were carried out in the four countries before launching the larger multicountry study . It was decided that a case–control design should be adopted [16–18] and that the ideal target sample size in each country would be in the range of 500 symptomatic patients and 500 asymptomatic controls. After obtaining all the necessary ethics approvals (August 2013–January 2014), extensive study document and site preparations, and GCP/GCLP training of the study site staff, the first patient was recruited in Nepal on July 30, 2014.
Interestingly, the recruitment of patients and controls showed considerable heterogeneity across countries. Fig 2 displays the country-specific enrolment over time. In Mali, where almost exclusively patients with persistent abdominal pain (and very few with persistent diarrhea) were recruited, the numbers of eligible patients and controls were high, and the recruitment went smoothly and without major obstacles. Indeed, a total of 553 patients and 553 matched controls were enrolled by May 2015, when the study was deemed successfully completed in Mali.
A different pattern was observed in Nepal. Patient recruitment started well in July and August 2014 but was temporarily interrupted for a period of almost three months because of protracted strikes and civil unrest that limited freedom of movement of the people. Later, patient enrolment resumed at a stable recruitment pace (with some seasonality seen) until August 2015, when nationwide strikes again affected the number of enrolled patients such that the initially targeted sample size of 500 symptomatic patients and 500 controls could not be reached.
A slow recruitment rate was observed in Côte d’Ivoire during the first months of the study, which was mainly explained by the fact that few eligible patients presented to the study hospital, whereas higher numbers of patients with either persistent diarrhea or persistent abdominal pain were reported from rural, remote health care posts in the surroundings of the study hospital. Hence, the recruitment was changed from a mainly passive approach—i.e., waiting for symptomatic patients to present to the study hospital—to a more proactive enrolment strategy, which included informing health care workers in the peripheral health care institutions and villages about the purpose and scope of the study so that they could actively refer potentially eligible patients to the study team.
A similar proactive approach was adopted in Indonesia, but the number of enrolled patients with persistent digestive disorders remained very low. Hence, after five months (August 2014–January 2015), it was decided by the study country team and the TMG to interrupt the study prematurely and to modify the design. Subsequently, a large community-based survey was carried out in the study area—which employed the same suite of diagnostic methods as foreseen in the original study protocol—to improve the diagnosis of parasitic infections and to increase the understanding of the causes and the clinical features that are related to these infections in rural Indonesia. The respective ethics committees were readily informed about the necessary modifications, and approval was obtained before the start of the survey.
What Were Key Challenges During Patient Recruitment?
During the course of the NIDIAG digestive study, several challenges were encountered that underscore why it remains an ambitious goal to perform studies on neglected clinical syndromes in the tropics. First, most study investigators reported difficulties in recruiting patients with persistent diarrhea as defined by the World Health Organization (WHO)—i.e., any diarrhea lasting for at least 14 days . Indeed, while long-lasting diarrheal diseases were perceived as a relevant clinical problem in all four study countries, many patients with persistent diarrhea had previously either taken empiric treatment (i.e., antibiotics, antiparasitics, or traditional medicines) or consulted a health care professional (e.g., village health center or local pharmacy). In this context, it is important to note that antidiarrheal medication and anti-infective drugs are regularly available as over-the-counter drugs without prescription in many low- and middle-income countries and are thus easily accessible [19,20]. Such medical and pharmaceutical interventions probably led to partial, transitory improvement in some patients, but the digestive symptoms returned after variable periods of intermittence in a considerable proportion of study participants. However, such patients frequently failed to meet the actual inclusion criteria because the onset of diarrheal symptoms after 2 days without diarrhea was defined as a new episode , and some of them could thus not be included.
Second, diarrhea and abdominal pain are relatively vague, subjective symptoms, and their reported severity depends on the perceived concepts of illness of the individual patient . Previous studies have shown that the length of the recall period is inversely related to the accuracy and reliability of reported patient complaints and that less severe episodes of diarrhea can be up to 40% underreported, with a recall period of one week . Furthermore, there is evidence from resource-constrained settings that diarrhea is less likely to be reported as a perceived health problem if it is common in a specific area [23,24]. For obvious reasons, “persistent abdominal pain” is even more difficult to define and to objectively retrace. The application of a standardized medical examination as part of the routine workup in the NIDIAG digestive study was thus considered an essential tool to complement data gathering and to counterbalance inaccuracies obtained by self-assessed morbidity reports.
Third, the NIDIAG digestive study was noninterventional, and study site–specific factors had thus to be considered. In Nepal, for instance, there are considerable government-run mass drug administration (MDA) campaigns that may have led to a decrease of helminthiasis-associated diarrheal diseases. Indeed, biannual deworming with albendazole for children aged 12–59 months has been carried out along with vitamin A supplementation since 1999, and albendazole is given twice a year to children attending grades one through ten of all public and private schools. Since 2005, albendazole has also been administered along with diethylcarbamazine citrate (DEC) as part of the efforts to eliminate lymphatic filariasis, and Nepal is expected to complete six rounds of MDA in 2018. Finally, pregnant women are also given albendazole during their second trimester. Hence, nearly all age groups in Nepal may potentially receive this anthelmintic drug through one of these MDA campaigns, and it will be interesting to see whether the NIDIAG study results confirm a beneficial effect on helminth-associated persistent digestive disorders.
Fourth, the etiology of persistent diarrhea and persistent abdominal pain is manifold and not limited to infectious causes. Autoimmune disorders, inflammatory diseases, neoplasms, and specific food intolerances may, among others, also lead to persistent intestinal disorders . The researchers and clinicians on site experienced that this fact needs to be explained in detail to symptomatic patients without an identified pathogen, and further investigations or referral to additional diagnostics should be initiated promptly. Of note, the detection of a certain pathogen in a stool or urine sample does not necessarily mean that the patient’s symptoms are actually linked to this finding . As the proportion and quantity of encountered pathogens may vary considerably from one setting to another, the NIDIAG study employed a case–control approach to elucidate pathogen-specific attributable fractions to the syndrome of persistent digestive disorders [27,28].
Controls were recruited from the same study centers as the symptomatic patients and were further matched by age group and sex to the symptomatic cases. It proved to be difficult to obtain consent of individuals to act as controls because they were frequently wondering why someone would be interested in analyzing stool and urine samples of healthy individuals. While careful explanation—being particularly sensitive to potential cultural barriers—helped to clarify this, some individuals still considered stool samples as very personal, intimate specimens and refused to participate. Indeed, the Nepalese study sites observed that it was very difficult to include asymptomatic patients who were unrelated to a symptomatic patient, while it was relatively straightforward to obtain consent from friends, relatives, or neighbors of symptomatic cases with persistent diarrhea, potentially because these individuals shared similar exposure characteristics (e.g., source of drinking water) with the symptomatic patient and wanted to know whether they were found to carry the same intestinal pathogens.
Have Challenges of Specific Laboratory Techniques Been Observed, and Which Solutions Have Been Adopted?
Two main areas of laboratory-associated challenges were identified during the NIDIAG digestive study: (i) general infrastructure and equipment and (ii) the conduct of the diagnostic laboratory tests. First, it was noted during the pre-study field visits that the available laboratory infrastructure in the study countries was, at most sites, not sufficient to accommodate such a broad diagnostic test panel as indicated by the NIDIAG study protocol . Indeed, there was no laboratory that had routinely employed all of the methods before the onset of the study. Hence, workshops were held to familiarize study staff with the techniques and to harmonize procedures across study countries. Additionally, many efforts were required to establish an internal and external quality assurance and monitoring system so that GCP/GCLP standards were met. It is important to note that such preparatory work is essential to ensure the proper conduct of scientific studies, but the significant human, logistic, and financial resources required are usually not considered sufficiently beforehand by both scientists and funding organizations . In Indonesia, the complete laboratory infrastructure had to be set up in a remote area of an island where the study center was located, which required several weeks of construction work. A short video showing the different study procedures as well as the laboratory infrastructure before and after the onset of the NIDIAG study in Maluku Tengah can be found in the supplementary material of this manuscript (S34). Of note, even minor logistic issues may have a profound impact. In Côte d’Ivoire and Nepal, for example, the absence of a functioning ventilation and climate control system in the laboratories led to several short delays in recruitment and specimen processing, as stool samples are unpleasant to work with and the study was thus not initiated before the ventilation systems were fixed.
Study site investigators frequently reported that the volume of a single stool sample was smaller than the amount stipulated in the respective SOP and was thus not sufficient to perform all required laboratory tests. Indeed, an amount of approximate 80 g of stool was required to run all tests (Table 2), as several diagnostic techniques (such as the Baermann funnel concentration for the diagnosis of Strongyloides stercoralis) require a large amount of stool. In such cases, a second sample had to be obtained from participants, but some, especially children, were not always able to provide the required amount and sometimes did not come back to provide the requested second sample.
Because of the host of different laboratory tests, the processing of a sample took approximately two to three hours, and it would have been ideal to collect stool specimens in the early morning and analyze them immediately in the laboratory. However, this was not always possible as the sites of patient enrolment were sometimes quite distant from the actual laboratory (e.g., the Dhankuta study site in Nepal) or the study laboratories had to send parts of the sample to reference laboratories (e.g., in Côte d’Ivoire and Mali, for stool bacteriology in case of persistent diarrhea). Such sample transfer had to be organized and carried out by couriers, such that laboratory staff sometimes worked until late in the evening when a sample arrived only in the afternoon.
With regard to difficulties associated with specific laboratory techniques, few deviations were observed, and the study investigators complied very well with the various SOPs. This observation was largely attributed to the two in-depth preparatory workshops that had taken place prior to the start of the study. Some of the remaining challenges experienced and potential solutions to overcome these are presented in Table 3.
As the NIDIAG project draws to an end, we feel that experiences and lessons learnt must be shared with the broader research community, clinicians, and disease control managers in countries where digestive disorders due to NTDs remain an important public health issue (Box 1). The current set of field-tested and ready-to-use SOPs has been implemented successfully in various sites, including those located in areas with restricted resources. We hope that providing open access to this large compilation of SOPs and highlighting key challenges met during the implementation and the conduct of the NIDIAG study on persistent digestive disorders might assist in further training and capacity building on all aspects of patient recruitment, clinical and diagnostic workup within GCP/GCLP standards, data management, and quality control, and thus improve the clinical diagnostic algorithms for patients suffering from NTDs. We look forward to experiences and lessons by other groups pursuing syndromic approaches to advance the diagnosis and clinical management of NTDs among neglected populations.
Box 1. Key Learning Points from the Multicountry NIDIAG Study on Persistent Digestive Disorders in the Tropics
- There is a need for studies investigating the etiology, diagnosis, and management of common clinical syndromes such as persistent digestive disorders in the tropics. Easily applicable, evidence-based diagnosis–treatment algorithms could substantially improve the clinical management of such syndromes at the primary health care level.
- The implementation of a quality assurance system is crucial for conducting multicenter clinical studies in resource-restricted settings, and it remains an ambitious goal to perform studies that fully comply with GCP and GCLP.
- Study documents such as SOPs and CRFs should be jointly developed and validated to harmonize procedures across study sites. Such field-tested tools are an important resource for researchers and health care staff and should thus be made publicly available.
- In a study on persistent digestive disorders, which was carried out by the NIDIAG consortium, perceived challenges were the prolonged recall period (≥14 days) for the identification of individuals with persistent abdominal pain and persistent diarrhea and the considerable heterogeneity seen regarding the recruitment pace of patients and controls in the four study countries.
S3 Clinical SOP. Selection of controls without digestive syndrome.
S4 Clinical SOP. Specific treatment procedures (including dosage).
S5 Clinical SOP. Assessing inclusion and exclusion criteria.
S6 Clinical SOP. Patient recruitment and patient flow.
S1 Laboratory SOP. Kinyoun staining technique (used in Côte d’Ivoire, Mali, and Nepal).
S2 Laboratory SOP. Modified acid-fast staining technique (used in Indonesia).
S3 Laboratory SOP. Crypto/Giardia Duo-Strip rapid diagnostic test (RDT).
S4 Laboratory SOP. Kato-Katz thick smear technique.
S5 Laboratory SOP. Baermann funnel concentration technique.
S7 Laboratory SOP. How to obtain a stool sample.
S8 Laboratory SOP. Formalin-ether concentration technique.
S10 Laboratory SOP. Direct fecal smear technique.
S11 Laboratory SOP. Preparation of aliquots for molecular post-hoc testing.
S12 Laboratory SOP. Urine point-of-care circulating cathodic antigen (POC-CCA) RDT for the diagnosis of Schistosoma mansoni.
S2 Quality SOP. Numbering system to be used in NIDIAG studies.
S3 Quality SOP. Management of study documents.
S6 Quality SOP. Internal quality control activities.
S7 Quality SOP. Good clinical laboratory practice (GCLP) supervision visits.
S10 Quality SOP. Handling of expired and disqualified products.
S11 Quality SOP. Handling and storage of rapid diagnostic tests (RDTs).
S1 Data management SOP. Completing case report forms (CRFs).
S2 Data management SOP. Procedure for data management.
- 1. Guerrant RL, Van Gilder T, Steiner TS, Thielman NM, Slutsker L, Tauxe RV, et al. Practice guidelines for the management of infectious diarrhea. Clin Infect Dis. 2001;32: 331–51. pmid:11170940
- 2. WHO. Diarrhoeal disease. Fact sheet no. 330. August 2009. Geneva: World Health Organization; 2009.
- 3. Becker SL, Vogt J, Knopp S, Panning M, Warhurst DC, Polman K, et al. Persistent digestive disorders in the tropics: causative infectious pathogens and reference diagnostic tests. BMC Infect Dis. 2013;13: 37. pmid:23347408
- 4. Abba K, Sinfield R, Hart CA, Garner P. Pathogens associated with persistent diarrhoea in children in low and middle income countries: systematic review. BMC Infect Dis. 2009;9: 88. pmid:19515227
- 5. Pawlowski SW, Warren CA, Guerrant R. Diagnosis and treatment of acute or persistent diarrhea. Gastroenterology. 2009;136: 1874–86. pmid:19457416
- 6. Ross AGP, Olds GR, Cripps AW, Farrar JJ, McManus DP. Enteropathogens and chronic illness in returning travelers. N Engl J Med. 2013;368: 1817–25. pmid:23656647
- 7. Yansouni CP, Bottieau E, Chappuis F, Phoba MF, Lunguya O, Ifeka BB, et al. Rapid diagnostic tests for a coordinated approach to fever syndromes in low-resource settings. Clin Infect Dis. 2012;55: 610–1. pmid:22563019
- 8. Yansouni CP, Bottieau E, Lutumba P, Winkler AS, Lynen L, Büscher P, et al. Rapid diagnostic tests for neurological infections in central Africa. Lancet Infect Dis. 2013;13: 546–58. pmid:23623369
- 9. Polman K, Becker SL, Alirol E, Bhatta NK, Bhattarai NR, Bottieau E, et al. Diagnosis of neglected tropical diseases among patients with persistent digestive disorders (diarrhoea and/or abdominal pain ≥14 days): a multi-country, prospective, non-experimental case–control study. BMC Infect Dis. 2015;15: 338. pmid:26282537
- 10. Dixon JR Jr. The International Conference on Harmonization Good Clinical Practice guideline. Qual Assur. 1998;6: 65–74. pmid:10386329
- 11. Ezzelle J, Rodriguez-Chavez IR, Darden JM, Stirewalt M, Kunwar N, Hitchcock R, et al. Guidelines on good clinical laboratory practice: bridging operations between research and clinical research laboratories. J Pharm Biomed Anal. 2008;46: 18–29. pmid:18037599
- 12. Thoma A, Farrokhyar F, McKnight L, Bhandari M. Practical tips for surgical research: how to optimize patient recruitment. Can J Surg. 2010;53: 205–10. pmid:20507795
- 13. Adams M, Caffrey L, McKevitt C. Barriers and opportunities for enhancing patient recruitment and retention in clinical research: findings from an interview study in an NHS academic health science centre. Health Res Policy Syst. 2015;13: 8. pmid:25971302
- 14. Bell-Syer SE, Thorpe LN, Thomas K, Macpherson H. GP participation and recruitment of patients to RCTs: lessons from trials of acupuncture and exercise for low back pain in primary care. Evid Based Complement Alternat Med. 2011;2011: 687349. pmid:18955352
- 15. Becker SL, Chatigre JK, Gohou JP, Coulibaly JT, Leuppi R, Polman K, et al. Combined stool-based multiplex PCR and microscopy for enhanced pathogen detection in patients with persistent diarrhoea and asymptomatic controls from Côte d’Ivoire. Clin Microbiol Infect. 2015;21: 591.e1–10.
- 16. Becker SL, Chappuis F, Polman K, N'Goran EK, von Müller L, Utzinger J. Reply: Epidemiologic studies need asymptomatic controls. Clin Microbiol Infect. 2015;21: e53–4. pmid:25964150
- 17. Dubourg G, Fenollar F. Epidemiologic studies need asymptomatic controls. Clin Microbiol Infect. 2015;21: e51–2. pmid:25964155
- 18. Raoult D. New diagnostic techniques highlight the need for negative controls. Clin Infect Dis. 2016;62: 809.
- 19. Mukonzo JK, Namuwenge PM, Okure G, Mwesige B, Namusisi OK, Mukanga D. Over-the-counter suboptimal dispensing of antibiotics in Uganda. J Multidiscip Healthc. 2013;6: 303–10. pmid:23990728
- 20. Ekwochi U, Chinawa JM, Osuorah CD, Odetunde OI, Obu HA, Agwu S. The use of unprescribed antibiotics in management of upper respiratory tract infection in children in Enugu, South East Nigeria. J Trop Pediatr. 2014;60: 249–52. pmid:24436341
- 21. Weiss MG. Cultural models of diarrheal illness: conceptual framework and review. Soc Sci Med. 1988;27: 5–16. pmid:3062796
- 22. Alam N, Henry FJ, Rahaman MM. Reporting errors in one-week diarrhoea recall surveys: experience from a prospective study in rural Bangladesh. Int J Epidemiol. 1989;18: 697–700. pmid:2807676
- 23. Kemper JT. The effects of asymptomatic attacks on the spread of infectious disease: a deterministic model. Bull Math Biol. 1978;40: 707–18. pmid:743565
- 24. Webair HH, Bin Ghouth AS. ‘This diarrhoea is not a disease…’ local illness concepts and their effects on mothers' health seeking behaviour: a qualitative study, Shuhair, Yemen. BMC Public Health. 2014;14: 581. pmid:24920306
- 25. Thomas PD, Forbes A, Green J, Howdle P, Long R, Playford R, et al. Guidelines for the investigation of chronic diarrhoea, 2nd edition. Gut. 2003;52 Suppl 5: v1–15. pmid:12801941
- 26. Platts-Mills JA, Liu J, Houpt ER. New concepts in diagnostics for infectious diarrhea. Mucosal Immunol. 2013;6: 876–85. pmid:23881355
- 27. Blackwelder WC, Biswas K, Wu Y, Kotloff KL, Farag TH, Nasrin D, et al. Statistical methods in the Global Enteric Multicenter Study (GEMS). Clin Infect Dis. 2012;55 Suppl 4: S246–53. pmid:23169937
- 28. Platts-Mills JA, McCormick BJ, Kosek M, Pan WK, Checkley W, Houpt ER. Methods of analysis of enteropathogen infection in the MAL-ED Cohort Study. Clin Infect Dis. 2014;59 Suppl 4: S233–8. pmid:25305292
- 29. Murray-Kolb LE, Rasmussen ZA, Scharf RJ, Rasheed MA, Svensen E, Seidman JC, et al. The MAL-ED cohort study: methods and lessons learned when assessing early child development and caregiving mediators in infants and young children in 8 low- and middle-income countries. Clin Infect Dis. 2014;59 Suppl 4: S261–72. pmid:25305296