https://orcid.org/0000-0002-1026-1082
Figures
Abstract
Objective
To assess whether an adapted Demographic and Health Survey (DHS) like cross-sectional household survey with full pregnancy histories can demonstrate the validity of health and demographic surveillance (HDSS) data by producing similar population structural characteristics and childhood mortality indicators at two HDSS sites in The Gambia–Farafenni and Basse.
Methods
A DHS-type survey was conducted of 2,580 households in the Farafenni HDSS, and 2,907 in the Basse HDSS. Household members were listed and pregnancy histories obtained for all women aged 15–49. HDSS datasets were extracted for the same households including residency episodes for all current and former members and compared with the survey data. Neonatal (0–28 days), infant (<1 year), child (1–4 years) and under-5 (< 5 years) mortality rates were derived from each source by site and five-year periods from 2001–2015 and by calendar year between 2011 and 2015 using Kaplan–Meier failure probabilities. Survey-HDSS rate ratios were determined using the Mantel-Haenszel method.
Results
The selected households in Farafenni comprised a total population of 27,646 in the HDSS, compared to 26,109 captured in the household survey, implying higher coverage of 94.4% (95% CI: 94.1–94.7; p<0.0001) against a hypothesised proportion of 90% in the HDSS. All population subgroups were equally covered by the HDSS except for the Wollof ethnic group. In Basse, the total HDSS population was 49,287, compared to 43,538 enumerated in the survey, representing an undercount of the HDSS by the survey with a coverage of 88.3% (95% CI: 88.0–88.6; p = 1). All sub-population groups were also under-represented by the survey. Except for the neonatal mortality rate for Farafenni, the childhood mortality indicators derived from pregnancy histories and HDSS data compare reasonably well by 5-year periods from 2001–2015. Annual estimates from the two data sources for the most recent quinquennium, 2011–2015, were similar in both sites, except for an excessively high neonatal mortality rate for Farafenni in 2015.
Conclusion
Overall, the adapted DHS-type survey has reasonably represented the Farafenni HDSS database using population size and structure; and both databases using childhood mortality indicators. If the hypothetical proportion is lowered to 85%, the survey would adequately validate both HDSS databases in all considered aspects. The adapted DHS-type sample household survey therefore has potential for validation of HDSS data.
Citation: Jasseh M, Rerimoi AJ, Reniers G, Timæus IM (2022) Assessment of the consistency of health and demographic surveillance and household survey data: A demonstration at two HDSS sites in The Gambia. PLoS ONE 17(7): e0271464. https://doi.org/10.1371/journal.pone.0271464
Editor: Orvalho Augusto, University of Washington, UNITED STATES
Received: December 16, 2021; Accepted: June 30, 2022; Published: July 13, 2022
Copyright: © 2022 Jasseh et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data from the Farafenni Health and Demographic Surveillance System (FHDSS) are available on ishare2- INDEPTH repository - "Farafenni INDEPTH Core Dataset 1982-2015 (Release 2018), which is provided by the INDEPTH Network Data Repository. Users may access this data by following the URL: www.indepth-network.org, DOI:10.7796/INDEPTH.GM011.CMD2015.v2. Data from the Basse Health and Demographic Surveillance System (BHDSS) and the DHS-type household survey data are available from the Medical Research Council Unit The Gambia at London School of Hygiene and Tropical Medicine (MRCG@LSHTM) through the Scientific Coordinating Committee (scc@mrc.gm) and subject to approval by the Joint MRCG/Gambia Government Ethics Committee for researchers who meet the criteria for access to confidential data.
Funding: The Farafenni and Basse HDSS sites are supported by Medical Research Council Unit The Gambia at London School of Hygiene and Tropical Medicine (MRCG@LSHTM). The cross-sectional sample household survey was supported by Nagasaki University (NU), Japan, through an agreement, MRC Internal Reference number 506518. Both MRCG@LSHTM and NU had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Despite the significant methodological improvements made in generating demographic information in most low and middle-income countries (LMICs), gaps remain in terms of quality and periodicity of existing data collection methods. Thus, they remain inadequate to support world class continuous scientific and socio-economic investigations to inform policy and influence practices for improving quality of life. In the absence of functional national vital registration systems, public health researchers in the global south have over the past two decades become increasingly dependent on health and demographic surveillance systems (HDSSs) for measurement of the impact of controlled interventions in communities and deriving reliable demographic indicators of interest, which can only otherwise be obtained from periodic cross-sectional Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS), or censuses. As a result, HDSS sites in sub-Saharan Africa have been instrumental in contributing evidence to demonstrate progress and achievements against the millennium development goals (MDGs) for instance, especially MDG4 [1, 2].
Despite criticisms levelled against them regarding their unrepresentativeness of national populations [3] and counter arguments in their favour [4, 5], dependence on these platforms for health and socio-economic research will continue for the foreseeable future. They will be key for measurement of targets associated with Sustainable Development Goal 3 (ensuring healthy lives). They also constitute the only appropriate resource to support medical research and large field trials in most LMICs that require precise numbers of cases and corresponding denominator counts for accurate measurement of incidence or prevalence rates of interest. But, like any other source of demographic information in LMICs, an HDSS is prone to both random and systematic errors. Most of these are usually identified and corrected using locally-designed check algorithms and data plausibility assessments. Moreover, other methodological simulations have demonstrated that HDSSs are sufficiently robust to withstand introduced random errors in excess of 20% [6]. Notwithstanding such internal integrity claims, HDSS databases need to be systematically and independently assessed and evaluated on a regular basis if they are to be relied on to generate evidence to influence policy in the southern world.
There are no set methods for independently assessing HDSS data for external and internal structural consistency. The traditional approach to HDSS data evaluation has been through an independent re-census of the entire demographic surveillance area, usually culminating in a dataset that can only be used to confirm household membership and population sizes of settlements and the entire surveillance area. The occurrence of key demographic events such as births and deaths, which are crucial to HDSS operations and measurement of demographic indicators and trends, cannot be ascertained by such a simple recount of residents. Other previous attempts to assess the quality of HDSS data and derived demographic indicators have focussed mainly on comparisons with data from Demographic and Health Survey (DHS) conducted in regions where the HDSS sites are located [7, 8]. As part of the recent “Every Newborn–INDEPTH” study (EN-INDEPTH) [9], HDSS data from five sites in Africa and Asia were used to assess data quality indicators for births and deaths data collected through a survey [10].
This objective of this study is to establish whether a sample DHS-type cross-sectional household survey conducted in a demographic surveillance area, collecting full pregnancy histories from women aged 15–49, can be used to assess routine HDSS data in terms of population counts and characteristics, and childhood mortality indicators. It compares household survey data with HDSS data from two demographic surveillance sites in The Gambia–the Farafenni and Basse Health and Demographic Surveillance Systems (FHDSS and BHDSS).
Methods
Study areas and populations
The Farafenni and Basse HDSS sites are located in middle and eastern Gambia respectively as shown in Fig 1. The Farafenni HDSS is described in detail elsewhere [11]. It is situated in the North Bank Region of The Gambia about 170 km from the capital city, Banjul; covering an area spanning 32km east and 22 km west of Farafenni Town. It was set up in October 1981 and initially comprised two clusters of 42 rural villages and hamlets located at least 10 km east and west of Farafenni town and a total population of about 12,000. The HDSS was expanded in 2002 to include Farafenni town and 23 settlements within a 5km radius of the town, referred to as the urban surveillance area. It prospectively follows up a total population of about 58,000 who are predominantly subsistence farmers with an annual average income per capita of less than UD$150 [11]. The site has supported cutting-edge medical, public health and demographic research since its inception; and three decades of surveillance in the area has documented significant declines in malaria transmission [12] and child mortality [1]. Surveillance was interrupted for a period of 13 months between 1st March 2008 and 31st March 2009.
A and B republished from [https://d-maps.com/m/africa/afrique/afrique10.gif] [13] and [https://d-maps.com/m/africa/west/west06.gif] [14] under a CC BY license, with permission from d-maps.com; C from https://commons.wikimedia.org/wiki/File:Gambia_districts.png [15].
The Basse HDSS covers all 224 villages and hamlets on the south bank of the Upper River Region, the eastern-most part of The Gambia. It covers an area of 1,111 sq Km and is located between 350 and 450 Km from the capital city. After an initial census conducted between September and November 2006 with an enumerated population of 136,387, routine demographic surveillance commenced in July 2007 [16, 17]. The population is presently about 190,000. Like in Farafenni, most residents are subsistence farmers with similar level of per capita income.
Both sites are characterised by high fertility levels with total fertility rates of more than 5 children per woman; and have large illiterate populations with more than 50% of females and over 40% of males aged 6 years and over in the administrative regions they are located in having never been to school [18]. Rural-urban migration is also a common feature in these rural regions. The 2013 National Population Census documented a net migration rate of -28% for the North Bank Region, and -13.4% for Upper River Region [19]. The main urban municipalities of Banjul and Kanifing serve as the main destinations for young males seeking employment opportunities, and females joining their husbands.
Health and demographic surveillance procedures
Surveillance procedures in Farafenni have been described in detail elsewhere [11]. The same procedures apply in Basse. Briefly, each system places four primary units under surveillance in their respective geographically demarcated areas and updates them accordingly in four-monthly intervals. These units are the settlements, compounds, households and individuals. A settlement comprises one or more compounds, which are usually demarcated by a fence; and a compound consists of one or more households. In both surveillance sites, a household is defined as a person or group of persons living in the same house or compound and sharing the same cooking arrangements. All settlements, compounds, households and individuals have unique identification numbers. New settlements, though rare, and compounds are enumerated as and when they appear; whilst the status of households and residency of individual members are updated during each four-monthly round of visits conducted by trained fieldworkers, who are predominantly male secondary school graduates, and aged between 20 and 45 years. The household head is the primary respondent during these updates, or an appropriate representative in his or her absence. Other household members are called on to provide other person-specific information such as pregnancy status and children’s vaccination records. In addition, resident village reporters recruited and trained on the recording of demographic events within their communities, keep records of births, deaths and migrations in and out of their settlements. This information is used by the fieldworkers to cross-check data collected during compound and household visits. The information was collected using household registration books up to 30th September 2015 in Farafenni and 29th February 2016 in Basse. Electronic data capture was then rolled out at both sites using computer tablets and customised software. Fieldworkers also record marriages and pregnancies. The latter are followed up to termination to confirm pregnancy outcomes and ascertain early neonatal and infant deaths.
Sample household survey
A DHS-type sample household survey was conducted between 10th December 2015 and 30th April 2016 by 10 enumerators in the Farafenni demographic surveillance area (DSA); and between 17th December 2015 and 4th June 2016 by 21 enumerators in the Basse DSA. Because of specific requirements of another study on neonatal mortality on which the surveys were based, half of all active households in the FHDSS and a quarter of households in the BHDSS as at 30th November 2015, evenly distributed across the respective surveillance areas, were randomly selected to participate. These represented a target of 2,862 households in Farafenni, and 2,780 in Basse. Only female enumerators, who were secondary school graduates in their early twenties, were recruited for cultural reasons to facilitate sufficient probing to distinguish between stillbirths and early neonatal deaths, as well as between abortions, miscarriages and stillbirths. None of the interviewers previously worked in the HDSS. The unique identification numbers of selected households were extracted from the HDSS databases and served as the only link between the survey and the HDSS to ensure independence between the two data sources. The same definition of household used in the HDSS was adopted by the enumerators for the sample survey.
The survey involved the administration of versions of the DHS household form, responded to by the household head or appropriate representative, and individual woman’s questionnaire adapted for use in this context [20]. In the household form, all resident members were listed and their relevant socio-demographic details recorded, including parental survival and residence. Individuals who had not spent two of the last three months in the household were excluded from further analysis. All women aged between 15 and 49 years listed in the household schedule were identified and a separate woman’s questionnaire administered to each woman personally. This included the standard DHS questions on reproduction, and detailed pregnancy and sibling histories. In obtaining the full pregnancy history of a woman, the enumerator asked about all pregnancies the woman had ever had, starting with the most recent and working back to the first pregnancy. For each pregnancy, the outcome (i.e. live birth, stillbirth, miscarriage or abortion) was recorded, together with the month and year of termination. In the case of live births, survival status of the child was established, with age at last birthday if still alive, or age at death if the child had died. All reported pregnancies that resulted in live births, and terminated between 1st January 2001 and 31st December 2015, are included in the analysis.
Data and statistical analysis
Since the assessment requires comparison of data from the same households, it was necessary to ensure that all households included in the survey could be linked to the correct household in the HDSS database using the unique household identification number, HHID. The HHID consists of eight digits in Farafenni, and eleven in Basse. A total of 2,652 households were surveyed in Farafenni and 2,993 in Basse and subjected to a systematic search to ensure that they correspond to the same households in the respective HDSS databases. Eventually, 72 households (2.7%) were dropped in Farafenni, 52 of which were with HHIDs that could not be reliably identified in the FHDSS database due to transcription error, and the rest were duplicates. The corresponding situation in Basse was 86 dropped households (2.9%), 74 not meeting the requirements for inclusion. Consequently, 2,580 households in Farafenni and 2,907 households in Basse were confirmed for inclusion in the analysis. The pregnancy histories of all women aged 15–49 in these households were collated and only those that terminated in live births were selected to generate a child-based dataset for each site for mortality estimation. No attempt was made to link mothers to their children between the two data sources.
The final lists of household identification numbers were also used to extract the relevant episodes and events associated with all household members, present and past, from the HDSS databases, censoring all episodes at 30th April 2016 in Farafenni and 4th June 2016 in Basse. These were regarded as the comparable HDSS datasets to be evaluated by the survey data in terms of population size, structure and characteristics, and childhood mortality levels and trends.
The survey and HDSS data were compared at two levels to ascertain the external and internal consistencies of the two data sources respectively. For the first level comparison, the coverage of the HDSS by the survey, both for total population counts and proportions of selected characteristics, was assessed to determine whether it adequately captured the HDSS population at the time of the fieldwork. The one-sample proportion test was used [21], applying the survey variable counts as sample size and the respective proportions they represent to the corresponding HDSS counts. A hypothesised proportion was required to perform the test, i.e. a threshold level considered to be the minimum acceptable coverage of the HDSS by the survey. None is recommended in the literature of HDSS and DHS methodologies. However, an a priori level of 90% was proposed and used in the analysis for two reasons. The first is the difference in operational definitions used in the two methods of data collection, especially with respect to residency status. For instance, household members who have been away for periods of up to four months and still considered residents by HDSS definition are likely to be missed in the count of residents in the household survey. Secondly, misapplication of the adopted definition of a household was more likely to divide unusually large households into smaller nuclear family units in the survey. Both situations result in an undercount of household members in the survey; and therefore, an overall coverage of at least 90% can be considered an acceptable capture of the HDSS in general. Therefore, with a survey coverage, p, of the HDSS and a hypothesised proportion of 0.9, a null hypothesis, H0: p≤0.9, was tested against an alternative hypothesis, HA: p>0.9 at 95% confidence level using the z-test statistic.
In the second or internal consistency comparison, the differences between the proportions of the selected population characteristics from both sources were compared using the two-sample proportion test [21]. An assumed null hypothesis of H0: ps = ph was tested against an alternative hypothesis of HA: ps≠ph at 95% confidence level using the z-test statistic, where ps is the survey proportion and ph the HDSS proportion. For the purpose of this analysis, broadly defined age groups by sex (i.e., 0, 1–4, 5–14, 15–29, 30–49, 50–64, and 65+) and ethnic group were selected as the population characteristics to focus on for the main reason that they are likely to be correctly reported in both the survey and HDSS.
The comparison of childhood mortality indicators derived from the two sources of data for both sites was based on calendar year estimates covering the 15-year period between 2001 and 2015. Childhood mortality indicators and rate ratios were estimated using survival analysis in Stata 17 [22]. Mortality rates from each data source were calculated as the number of deaths per thousand live births and were determined for each period and year using Kaplan–Meier failure probabilities for each of the age groups of interest that constitute measures of childhood mortality. These are defined as: (i) deaths within the first month of life (i.e. <1 month) for neonatal mortality; (ii) deaths in the first year of life (i.e. <1 year) for infant mortality; (iii) deaths between exact ages one and four years (i.e. 1–4 years) for child mortality; and (iv) deaths between birth and exact age five years (<5 years) for under-5 mortality. For each age group, period and recent year survey-HDSS rate ratios were determined using the Mantel-Haenszel method and p-values reported at 95% confidence level.
Ethics statement
The Farafenni and Basse HDSSs have approval from the Joint MRCG/Gambia Government Ethics Committee to conduct continued surveillance. The same committee approved the instruments used in the household surveys at both sites, and for which consent was sought and documented from household heads and eligible women prior to the administration of the respective questionnaires.
Results
Comparison of population indicators
The household size characteristics of the sampled households in both sites are presented in Table 1. In each of the sites, the two data sources show similar size characteristics, with households tending to be much larger in Basse than Farafenni. Compared with the HDSS, the survey undercounted the sizes of 13% and 18% of households in Farafenni and Basse, respectively. Collectively, the Farafenni households yielded a total population of 27,646 in the HDSS, compared to 26,109 captured in the household survey, implying a coverage of 0.944 (95% CI: 0.941–0.947) (see Table 2). Applying the one-sample proportion test against the hypothesised proportion of 0.9, there is no evidence to suggest that the survey in Farafenni inadequately represented the FHDSS in terms of the population count (p<0.001). Similar evidence was obtained for the population characteristics of sex and ethnic group, except for Wollof group, for who the survey’s coverage of the HDSS was 0.896 (95% CI: 0.890–0.902) and fell just short of the desired threshold.
In Basse, the selected households accounted for a total population of 49,287 in the HDSS and 43,538 covered by the survey, giving a coverage of 0.883 (95% CI: 0.880–0.886). This is less than the hypothesised proportion and the one-sample proportion test confirmed that the household survey in Basse yielded an undercount of the BHDSS in terms of population size (p = 1). All population sub-groups in the HDSS were inadequately represented by the survey according to the derived statistical evidence.
In terms of age structure by sex, the survey reported more than double the number of infants (i.e. <1 year) for both sexes than the HDSS in Farafenni; and more than three times for the same age group in Basse. Due to enumerator error for not confirming the date of birth field in the electronic data capture form that was defaulted to current date, dates of birth of many individuals were reported to be similar to the date of interview, thus causing the unexpected excess number of infants aged less than 1 year. Apart from this anomaly, the survey significantly undercounted all defined age groups of the population for both sexes in Basse. In Farafenni, however, only the age groups 15–29 and 30–49 for males, and 30–49 and 65+ for females manifested statistically significant evidence of undercounts.
The two-sample proportion tests investigating revealed statistically significant differences in sample proportions at the 0.05 level for the Wollof ethnic group in Farafenni (p = 0.003); and Mandinka and Sarahule groups in Basse (p = 0.041 and <0.001, respectively). The sex distributions in both data sources were similar at both sites despite the overall undercount reported for Basse. However, structural differences were observed for the age group 15–29 among males (p = 0.008) and 30–49 among females (p = 0.004) in Farafenni; and the two age groups 5–14 and 15–29 among males (p = 0.035 and <0.001, respectively), and 15–29 and 30–49 among females (p = 0.021 and <0.001, respectively) in Basse.
Comparison of childhood mortality indicators
The pregnancy histories yielded 12,528 live births from 6,239 women aged 15–49 in Farafenni; and 11,106 live births from 9,793 women of the same age group in Basse. The corresponding number of women aged 15–49 in the comparative HDSS datasets were 7,168 and 12,833 respectively. The childhood mortality indicators and rate ratios derived from HDSS and household survey data are presented in Table 3 by site and 5-year periods. In Farafenni, the survey-derived estimates of neonatal mortality were consistently higher than HDSS-based rates throughout the 15-year period. A similar trend was observed for infant mortality estimates, save that the rates in the earliest period were not significantly different from each other with a rate ratio of 1.23 (95% CI: 0.93–1.63). Estimates of child mortality rates from the two data sources were statistically similar for all three periods. In the case of Basse, there were no statistically significant differences between survey and HDSS derived childhood mortality indicators in the most recent 5-year period, which is the only period fully covered by the two data collection methods.
Similar childhood mortality estimates and rate ratios by site and calendar year are presented in Table 4 for the most recent 5-year period, 2011–2015. Whilst a similar trend of consistently higher neonatal mortality estimate from the household survey was observed in Farafenni, the reported rate ratios were statistically significant only for 2013 and 2015, i.e. 2.49 (95% CI: 1.27–4.83) and 6.76 (95% CI: 3.33–13.70), respectively. The relatively high neonatal mortality rate of 63.5 (95% CI: 49.2–81.7) obtained from the survey for Farafenni in 2015 also inflated the corresponding estimates for infant and under-5 mortality. It was only in 2015 that Farafenni recorded statistically different under-5 mortality rates between the two sources of data. In Basse, both data sources produced statistically similar childhood mortality indicators for each year, except for child mortality in 2012 (RR = 0.47; 95% CI: 0.23–0.95) and under-5 mortality in 2011 (RR = 0.64; 95% CI: 0.42–0.97).
Discussion
This study attempts to ascertain whether a DHS-type cross-sectional household survey can be used to assess the quality of data and reliability of a prospective demographic surveillance system. The focus in this attempt is limited to determining if the two different methods of data collection applied to the same population yield similar measures in terms of size, structure and commonly derivable childhood mortality indicators. These methods are based on distinctly different operational definitions, and have contrasting advantages and disadvantages associated with them by virtue of their designs. Such are their basic theoretical differences that, even if reporting and data collection were perfect, they would not yield the same datasets. Due consideration must therefore be taken of the key features of each method in the comparison of data independently generated by them with a view to ascertaining whether they adequately match each other, with one hence validating the other.
In terms of population size and structure, the adapted DHS-type survey sufficiently validates the HDSS data in Farafenni; whilst it fell short of covering the Basse HDSS assuming, a priori, a minimum acceptable coverage proportion of 0.9. This may be related to the fact that household sizes are larger in Basse than in Farafenni. Since the population sizes represent the total listed household members in the survey and the population in the HDSS as at the time of the household survey, differences between the two sources of data can only be attributed to their operational definitions of residency. Because the HDSS rosters are based on periodic visits, they may retain as active the residency status of a household member who is away from the household for up to four months. The cross-sectional household survey, in contrast, is fully up-to-date on the day of the survey but does not capture who was resident the last time the HDSS fieldworker visited the household.
The comparison of household size ranges from the survey against those indicated in the HDSS database shows that 517 (18%) households in Basse were under-enumerated at varying degrees, including 30 households with sizes in excess of 60 but reported to have only up to 40 members in the survey (Table 1). On the other hand, 238 (8%) were over-enumerated, but at lesser extent than those that were undercounted. This is evidently a contributory factor to the observed survey undercount in Basse. Some survey respondents misinterpreted the definition of a household provided by the enumerators to mean their immediate nuclear family and thereby listed far fewer members of the household than documented in the HDSS database. The sex structures from the two data sources in Basse also reveal that the undercount affected males more than females (Table 1). This is expected as a considerable proportion of men in this population are engaged in trade and more likely to travel outside the surveillance area than women.
The designs of the two data collection strategies also impact on estimates of childhood mortality indicators differently. As in any DHS, pregnancy histories are characterised by recall bias whose magnitude increases with time since the termination of the pregnancy [23, 24]. Also, mis-statement of dates of birth and death of children by mothers can cause displacement within the respective age categories considered in the analysis, as well as across calendar periods [25]. For the prospective demographic surveillance systems, pregnancies and their outcomes, in particular pregnancies that end in early neonatal deaths, can be missed between rounds of data collection [26]. The result in Farafenni, therefore, that consistently higher neonatal mortality rates were obtained from the household survey than were derived from the HDSS data for the 15-year period considered in the analysis is not particularly surprising. It has been observed in another comparative study of these HDSS data and the first Gambian DHS data [27], as well as at other HDSS sites [10]. Notwithstanding the higher neonatal mortality rates from the household survey, the derived infant and under-5 mortality rates for the earlier period of 2001–2005 were statistically similar to those obtained from the HDSS data. The observed statistically significant differences in these rates for the two most recent periods can be attributed to two factors, each affecting one period. The first factor is the disruption of surveillance in Farafenni for 13 months between March 2008 and April 2009, which inevitably caused a significant number of neonatal deaths to be missed between 2008 and 2010, hence the low rate of 10.0 per 1,000 live births recorded for the period 2006–2010. The impact of this break is clearly demonstrated in the annual trends in neonatal, infant and child mortality rates from both data sources presented in Fig 2.
For the most recent period 2011–2015, a relatively high level of neonatal mortality was obtained from the survey for Farafenni, i.e. 63.5 per 1,000 live births compared to 10.7 per 1,000 live births from the HDSS. According to the pregnancy history data, it is the highest level of annual neonatal mortality rate ever recorded in Farafenni. However, in a community reported to have attained its MDG4 goal seven years before time [1], the reported level of neonatal mortality for 2015 in Farafenni is likely to be due to displacement error of births and deaths of neonates caused either by wrong reports by mothers or entry errors on the part of enumerators. The situation in Basse is completely different in that almost all annual childhood mortality indicators from both data sources are statistically similar (Table 4). This is further demonstrated in Fig 2.
Although the survey in Basse fell short of providing an acceptable coverage of the HDSS based on a hypothesised proportion of 0.9, the pregnancy histories have provided childhood mortality rates similar to those obtained from the HDSS for the period 2011–2015. Since it was shown that men were more likely to be missed in such surveys, and simulations have demonstrated that HDSS data relating to population structure and derivable mortality rates can withstand random errors of up to 20% [6], it is possible to reconsider lowering the assumed a priori hypothesised proportion to 0.85. If such a level were used in the analysis presented in this study, the household surveys in both Farafenni and Basse would have sufficiently represented the respective HDSS data in terms of population size and structure, and derived childhood mortality indicators would remain the same.
Whilst the results clearly demonstrate that an adapted DHS-type survey can be used to validate routine HDSS data using population structure and childhood mortality, there are other aspects of HDSS data that can be validated in a similar manner, such as estimates fertility. Ideally, we would have liked to compare the two data sources in terms of whether they identified similar groups of women, and whether the women reported similar childbearing histories, and whether the two sets of children have similar levels and trends of mortality. However, this extended analysis could not be undertaken due to the poor quality of date-of-birth information collected by the household survey. The field for date of birth in the electronic survey questionnaire was designed to default to “current date” or date of interview and to be set accordingly by the enumerator depending on the reported date of birth to be followed by the return key for confirmation of the date entered. However, due to enumerator error, entry of reported date of birth was not confirmed in many cases using the return key, hence resulting in dates of birth being the same as interview dates. This is responsible for the significantly exaggerated proportions aged 0 in both populations and for both sexes (see Table 2).
Considering the high cost of conducting such adapted DHS-type household surveys in a setting as resource intensive as the HDSS, further research will be required to establish the minimum representative sample size that will suffice to efficiently validate or assess HDSS data quality and database integrity on a regular basis. Whilst this study was based on sites with similar update round intervals of four months, similar trials of the method should be made at sites with update round intervals of six months and one year to ascertain the impact of data collection intervals on the suitability of the adapted DHS survey to assess HDSS data.
Acknowledgments
We are grateful to the two teams of female enumerators who conducted the household surveys in Farafenni and Basse. We also thank Pierre Gomez and Mamadi Sidibeh for the support and guidance they accorded to the enumerators, especially in the identification of households.
References
- 1. Jasseh M., et al., Reaching millennium development goal 4—the Gambia. Trop Med Int Health, 2011. 16(10): p. 1314–25. pmid:21707875
- 2. Trape J.F., et al., New malaria-control policies and child mortality in senegal: reaching millennium development goal 4. J Infect Dis, 2012. 205(4): p. 672–9. pmid:22238469
- 3. Byass P., Person, place and time—but who, where, and when? Scand J Public Health, 2001. 29(2): p. 84–6. pmid:11484870
- 4. Bocquier P., Sankoh O., and Byass P., Are health and demographic surveillance system estimates sufficiently generalisable? Glob Health Action, 2017. 10(1): p. 1356621. pmid:28820344
- 5. Byass P., et al., Lessons from history for designing and validating epidemiological surveillance in uncounted populations. PLoS One, 2011. 6(8): p. e22897. pmid:21826215
- 6. Fottrell E., Byass P., and Berhane Y., Demonstrating the robustness of population surveillance data: implications of error rates on demographic and mortality estimates. BMC Med Res Methodol, 2008. 8: p. 13. pmid:18366742
- 7. Byass P., et al., DSS and DHS: longitudinal and cross-sectional viewpoints on child and adolescent mortality in Ethiopia. Popul Health Metr, 2007. 5: p. 12. pmid:18162133
- 8. Deribew A., et al., Under-five mortality rate variation between the Health and Demographic Surveillance System (HDSS) and Demographic and Health Survey (DHS) approaches. BMC Public Health, 2016. 16(1): p. 1118. pmid:27776500
- 9. Baschieri A., et al., "Every Newborn-INDEPTH" (EN-INDEPTH) study protocol for a randomised comparison of household survey modules for measuring stillbirths and neonatal deaths in five Health and Demographic Surveillance sites. J Glob Health, 2019. 9(1): p. 010901. pmid:30820319
- 10. Nareeba T., et al., Neonatal and child mortality data in retrospective population-based surveys compared with prospective demographic surveillance: EN-INDEPTH study. Popul Health Metr, 2021. 19(Suppl 1): p. 7. pmid:33557871
- 11. Jasseh M., et al., Health & Demographic Surveillance System Profile: Farafenni Health and Demographic Surveillance System in The Gambia. Int J Epidemiol, 2015. 44(3): p. 837–47. pmid:25948661
- 12. Ceesay S.J., et al., Changes in malaria indices between 1999 and 2007 in The Gambia: a retrospective analysis. Lancet, 2008. 372(9649): p. 1545–54. pmid:18984187
- 13. Dalet D., Blank political map of Africa, in d-maps.com: free maps, free blank maps, free outline maps, free base maps, D. Dalet, Editor. 2022, d-maps.com: https://d-maps.com/.
- 14. Dalet D., Blank political map of West Africa, in d-maps.com: free maps, free blank maps, free outline maps, free base maps, D. Dalet, Editor. 2022, d-maps.com: https://d-maps.com/.
- 15.
Rarelibra, Map of the districts of The Gambia. 2006: https://commons.wikimedia.org/.
- 16. Bojang K.A., et al., Two strategies for the delivery of IPTc in an area of seasonal malaria transmission in the Gambia: a randomised controlled trial. PLoS Med, 2011. 8(2): p. e1000409. pmid:21304921
- 17. Mackenzie G.A., et al., Monitoring the introduction of pneumococcal conjugate vaccines into West Africa: design and implementation of a population-based surveillance system. PLoS Med, 2012. 9(1): p. e1001161. pmid:22272192
- 18.
Gambia Bureau of Statistics (GBoS) and ICF, The Gambia Demographic and Health Survey 2019–20. 2021, GBoS and ICF: Banjul, The Gambia and Rockville, Maryland, USA.
- 19.
Gambia Bureau of Statistics, 2013 Population and Housing Census Report, Volume 4: National Migration Analysis. 2015, Gambia Bureau of Statistics: Banjul, The Gambia. p. 40.
- 20. International ICF, Demographic and Health Surveys Methodology—Questionnaires: Household, Woman’s, and Man’s., in Measure DHS Phase III. 2011: Calverton, Maryland, USA.
- 21.
Sabo R. and Boone E., Statistical research methods: a guide for non-statisticians. 2013: Springer Science+Business Media, New York. ix, 214 pages.
- 22.
StataCorp., Stata: Release 17. Statistical Software. 2021, StataCorp LP: College Station, TX.
- 23.
Pullum T.W. and Becker S., Evidence of omission and displacement in DHS birth histories, in DHS Methodological Reports No. 11. 2014, ICF International: Rockville, Maryland, USA.
- 24.
MacQuarrie K.L.D., et al., Consistency of reporting of terminated pregnancies in DHS calendars, in DHS Methodological Reports No. 25. 2018, ICF: Rockville, Maryland, USA.
- 25.
Pullum T.W. and Staveteig S., An assessment of the quality and consistency of age and date reporting in DHS Surveys, 2000–2015, in DHS Methodological Report No. 19. 2017, ICF: Rockville, Maryland, USA.
- 26. Eilerts H., et al., Age patterns of under-5 mortality in sub-Saharan Africa during 1990–2018: A comparison of estimates from demographic surveillance with full birth histories and the historic record. Demographic Research, 2021. 44: p. 415–442. pmid:35368427
- 27. Rerimoi A.J., et al., Under-five mortality in The Gambia: Comparison of the results of the first demographic and health survey with those from existing inquiries. PLoS One, 2019. 14(7): p. e0219919. pmid:31335884