https://orcid.org/0000-0002-2923-0526
Figures
Abstract
Introduction
Effective policymaking relies on high-quality data to understand social contexts, identify target populations, and evaluate interventions. In low and middle-income countries (LMICs), household surveys often fill data gaps, providing insights into social dynamics and policy impacts. In Pakistan, the Pakistan Demographic Health Survey (PDHS) and Pakistan Social and Living Standards Measurement (PSLM) are crucial sources of information. While both surveys cover health and socioeconomic indicators, their methodologies and questionnaires vary, leading to potential discrepancies in data.
Methods
This paper compares PDHS 2017-18, PSLM 2018-19 (provincial level) and PSLM 2019-20 (district level) using family planning and child immunization modules as examples. Similar indicators under each section are examined for differences using weighted proportion t-test. For family planning, we analyzed PDHS 2017-18 and PSLM 2018-19 because PSLM 2019-20, doesn’t have family planning section. For immunization, we analyzed PDHS 2017-18, PSLM 2018-19 and PSLM 2019-20.
Results
Analysis reveals high concordance in family planning indicators with differences of within two percent. Differences in the rates of BCG which is given at birth are under one percent and for the first dose of pentavalent vaccine are near one percent. However, the differences start diverging thereafter and are near nine percent for dose 3 of the pentavalent vaccine. There is high level of concordance between the results of the provincial and district PSLM surveys conducted one year apart.
Conclusion
We describe the differences and relative similarities of the PSLM and PDHS surveys, as means to better incorporate their evidence in policy decisions. Both PSLM and PDHS serve a slightly different niche in that PDHS provides more in depth understanding of family planning whereas PSLM connects many health and social indicators with economic measures and gives granularity at the district level. However, to enhance the confidence of policymakers in both the surveys, we describe their concordance and differences and how they may be used in policy decisions.
Citation: Ibrahim M, Farman N, Rehman HU, Abdullah M, Cheema AM, Aamir M, et al. (2025) Towards Cohesive National Surveys in Pakistan: A Comparative Study of DHS and PSLM. PLoS ONE 20(3): e0320044. https://doi.org/10.1371/journal.pone.0320044
Editor: Naveed Sadiq, Khyber Medical University, PAKISTAN
Received: October 19, 2024; Accepted: February 13, 2025; Published: March 25, 2025
Copyright: © 2025 Ibrahim et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The Demographic and Health Survey (DHS) data can be accessed through the DHS Program website at https://dhsprogram.com/Countries/Country-Main.cfm?ctry_id=31. The Pakistan Social and Living Standards Measurement (PSLM) data is available on the Pakistan Bureau of Statistics website at https://www.pbs.gov.pk/content/microdata. Access to these datasets may require registration or adherence to specific terms and conditions as specified by the respective data providers.
Funding: The manuscript has been prepared by a multidisciplinary team with expertise in public health, epidemiology, and economics. This research was supported by the Bill & Melinda Gates Foundation [grant number: INV-051108], The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist
1. Introduction
Effective evidence based policymaking depends on the availability of high-quality data, which can illuminate social contexts, identify appropriate populations to target with interventions, and enable timely evaluation of policies and programs. Conversely, unreliable data with inaccuracies, incompleteness, or the absence of essential administrative records such as vital statistics and public service outputs, limits the understanding of social issues and the impact of interventions [1,2]. Unreliable data undermines policymakers’ ability to gauge the effectiveness of their initiatives, potentially leading to inefficient or futile public sector investments. In many low and middle-income countries (LMIC), where program data are unreliable, household surveys are needed to bridge gaps in program data [3–6], to provide insights into social dynamics and track policy and program impacts over time [7–10].
As a developing country, Pakistan relies on a series of surveys that inform various aspects from health, social issue, economics and more (https://www.pbs.gov.pk/). Of these, the leading ones are the Pakistan Demographic Health Survey (PDHS) and Pakistan Social and Living Standards Measurement (PSLM). The PDHS survey aligns with international Demographic Health Surveys (DHS) indicators (https://dhsprogram.com), while the PSLM is based on the World Bank-supported Living Standards Measurement Study (https://www.worldbank.org/en/programs/lsms/overview).
The PDHS includes separate questionnaires for households, ever-married women, ever-married men, biomarkers, and community. It provides extensive details on health indicators and is conducted at the provincial level, taking place sporadically, with intervals of at least five years [11]. It’s sampled at the provincial level and includes around 16,000 married women of reproductive age and 4,000 married men.
On the other hand, the PSLM covers a wider range of indicators beyond health and is conducted concomitantly with the Household Integrated Economic Survey (HIES), focusing on income and expenditures from the same household. The PSLM has alternates between provincial and district levels surveys in biennial cycles [12,13]. The provincial PSLM includes around 25,000 respondents who are asked in depth about health and social indicators. The sample for the district version is around 80,000 but with fewer questions across any domain and has fewer domains.
The Demographic Health Surveys and Social and Living Measurement Surveys are also available in many LMICs. The similarities in sampling and questions between the two surveys make them complementary for various health, social, and economic indicators. However, the extent of this complementarity or key differences has never been examined. Another potential option would be to use the relative strengths of these surveys to better inform about social indicators at national and provincial level and allow modeling to infer some indicators at the district levels. As a first step, we compare the PDHS and PSLM provincial surveys in Pakistan, using immunization, family planning, and women’s decision-making modules as examples. This comparison could serve as a model for these surveys conducted in other LMICs.
2. Methodology
2.1. Structure of the surveys
PDHS and PSLM (provincial and district) cover a diverse range of modules in their surveys (Table 1).
PDHS primarily focuses on health and demographics indicators, while PSLM includes a broader range of socio-economic indicators, including health, demographics, expenses, income, education, consumption, and wealth [14]. The PDHS and the provincial version of the PSLM surveys ask more in-depth questions from households (11,856 and 24,809 households, respectively), while the PSLM district version asks fewer questions from a much larger sample of households (176,790).
All three surveys cover basic demographic, education, and employment questions, along with child immunization, diarrhea among children, household characteristics (for estimating wealth index), pre and post-natal care, basic health service questions, and basic knowledge about tuberculosis (TB), Hepatitis B & C. PDHS uniquely captures child nutrition, weight, and height in their biomarker questionnaire, along with questions on domestic violence and gender roles. In contrast, PSLM surveys uniquely capture comprehensive income data, information on communication technology (ICT), and the food insecurity experience scale (FIES). The provincial PSLM survey covers household expenditure and income in depth to create a proper balance sheet, while the district version only includes a short section on income.
PDHS (2017–18) survey covers four provinces, two regions (AJK and GB), (erstwhile) FATA, and Islamabad. Meanwhile, the PSLM provincial-level survey includes four provinces of Pakistan, with FATA being part of KP. To standardize the analysis, we restricted the regions to four provinces (Punjab, Sindh, KP, and Balochistan) and included FATA in KP in PDHS to make it consistent with PSLM.
Surveys were compared in three stages: (1) briefly comparing modules covered by both surveys, (2) reporting questions addressed on family planning and child immunization in both surveys, and (3) comparing estimates of key indicators for family planning and child immunization.
This study utilized modules on ‘Reproduction’, ‘Contraception’, and ‘Child Immunization’ from PDHS and ‘Family Planning’, ‘Women in Decision Making’, and ‘Immunization’ from PSLM. However, the PSLM district-level survey does not include the ‘Family Planning’ section; hence, we focused on the ‘Immunization’ section for differential analysis.
The respondents for both surveys were Married Women of Reproductive Age (MWRAs) aged 15–49 years. The sampling strategy, provided by the Pakistan Bureau of Statistics, used the same methodology and enumeration blocks. Table 2 illustrates the rudiments of the two surveys based on the two modules.
Using the described survey modules, two new variables, ‘knowledge of family planning’ and ‘contraceptive method used,’ were created because the questions related to these variables were generic rather than specific to the contraceptive method used. There were minor differences in the categories of ‘source of the method used’ and ‘method type of family planning’ for PDHS and PSLM. The categories were combined to address this issue of source difference (Table 3). A similar exercise was performed while dealing with contraceptive methods. The analysis gauged the estimates for current users of modern and traditional methods.
The children’s sample in the immunization module was restricted to the age of 12–35 months. The age bracket for each vaccine Childhood TB, BCG, Pentavalent (Diphtheria, Tetanus, Pertussis, Hib Pneumonia and Meningitis, Hepatitis B), Polio (Polio-0, Polio-1, Polio-2, Polio-3 and Inactivated Polio vaccine (IPV)) and Measles-1 were assessed at 12–23 months, except for Measles-2 (marked at 24–35 months). PDHS records immunization data on two parameters, ‘yes, on record’ and ‘yes, on recall’ for all vaccines, whereas PSLM includes an additional category, ‘yes, through campaign’ for Polio and Measles vaccines. The primary objective of the analysis was to obtain the percentage of children immunized in both surveys without deviating from the actual number of vaccinated children; therefore, we did not drop or exclude the ‘yes, through campaign’ category.
2.2. Method of analysis
For the comparison analysis between PDHS and PSLM, we calculated the proportions of indicators using area weights in each survey to reduce bias in estimates. These weights also provide auxiliary data on population-known characteristics, thereby reducing sampling errors [15]. We estimated a simple difference between the indicators, followed by a two-tailed proportions test [16,17]. The null hypothesis stated that the difference between weight-adjusted estimates from PDHS and PSLM was equivalent to zero for each indicator of family planning and child immunization. The generic form of the test is given below:
As we are using secondary data for our analysis, institutional review board (ethics committee) approval was not required. These anonymized datasets are available on online portals.
3. Results
The PDHS contains more detailed questions on family planning, including the brand names of contraceptives, side effects, sterilization details, timeline of the last method used, and advice by healthcare workers on method use. In contrast, PSLM includes questions about satisfaction levels with the current family planning method (S1 data). Both PDHS and PSLM provincial-level surveys feature similar, in-depth questions about the child immunization module, whereas the PSLM district level survey only asks whether the child is immunized for the recommended vaccines or not. The subsequent subsections present the results for family planning and child from PDHS and PSLM datasets.
3.1. Family planning
There is high concordance between the results of the family planning modules from PDHS and PSLM provincial (which is the only version that asks about FP). For example, 99% of Married Women of Reproductive Age (MWRA) in PDHS and 98% in PSLM have heard of at least one family planning method, and 33% and 34% MWRAs were currently using a family planning method, with 26% and 29% using traditional methods, respectively. Similarly, PDHS shows that 46% use modern methods compared to 47% in the PSLM, with a similar method mix. Differences are < 4% for the ‘source of methods’ and < 6% for ‘who makes the decision to use FP’, though statistically significant due to large sample sizes. (Table 4)
The PDHS includes more family planning questions about the side effects, problems faced while using contraception, the number of times the respondent used a method to avoid or delay pregnancy, or ‘what year was the sterilization performed’. PSLM does not have those questions.
3.2. Child Immunization
Differences between PDHS and PSLM provincial-level surveys are more pronounced for immunization (Table 5). For example, differences are < 1% for BCG, which is given within a week after birth and is most likely to be remembered. The differences expand with increasing age of the child, as measured by the differences with each successive dose of pentavalent and pneumococcal vaccines, starting from around 1% for the first dose to just over 9% for the third dose. The differences are higher for measles and over 18% for IPV, which is given sporadically. Polio vaccine recall is distributed non-systematically, as the vaccine is given monthly during campaigns, leading to difficulty in distinguishing between regular and campaign doses. Overall, these differences lead to greater divergence between PDHS and PSLM for immunization at 24%, a full log order higher than the 2% seen for family planning. The divergence increases as the child grows older.
Compared to PDHS and PSLM provincial surveys, differences between provincial and district PSLM are minor. Most vaccine percentage differences are below 1% without any statistically significant difference. However, there is a statistically significant drop in polio vaccine coverage in the district PSLM 2019–20. The difference between basic immunization and all age-appropriate vaccinations (both including and excluding polio vaccines) is higher and increases from 2018–19 (provincial) to 2019–20 (district) PSLM.
4. Discussion
Our analysis indicates that while differences between rates in PSLM and PDHS have closed for family planning, and early immunization, the differences in immunization rates diverge after the 3rd month of the child’s age. While each survey serves a different niche, our comparison analysis allows the identification of data gaps that can help make these two distinct but complementary surveys more compatible and, in turn, more usable.
There was a 20-percentage points discrepancy between PDHS 2012 and the concurrent PSLM for contraceptive use. This issue was discussed with the survey team at the Pakistan Bureau of Statistics. Their internal review suggested that the discrepancy may have arisen because the interviewees for PSLM could be anyone in the household, not necessarily the MWRA, as is the case with PDHS. Persons other than the MWRA in the household may give different responses, resulting in misleading results [18]. This has since been rectified and is reflected in narrowing of the gap in family planning indicators between the two surveys in 2017–18. The same is true for early age immunization but begins to diverge after the third months of the child’s life, perhaps reflecting recall bias of the mother [19].
Family planning is simpler to recall for a woman to remember since she is the primary actor in FP usage, and the use is a one-off event, as only the last status before the survey is included. However, immunization is more complex in that several vaccines are administered, separately and concomitantly, over the first 24 months of a child’s life. This is further complicated if there are more than one child of vaccination age in the household. Additionally, routine polio doses and monthly supplemental campaigns are identical and indistinguishable for parents. This creates several data points for a mother to remember, adding to errors in recall [19]. So many vaccinations may be too much for most mothers to remember. As we found, recall is fully concordant between PDHS and PSLM for BCG (given at birth) and first dose of the Pentavalent vaccine (given at 6 weeks). Thereafter, divergence increases with the increasing age of the child and is the highest for polio vaccine, which continues to be given monthly until a child turns five years old [20]. The quality and internal consistency of the PSLM surveys is validated by the relatively similarity in results of the provincial and district surveys conducted one year apart (Table 5).
Global evidence suggests that having a written record a memory aid helps improve recall [21–23]. The situation in Pakistan has markedly improved since 2006, when only 10% households had an immunization card [24] in the PDHS 2017–18 survey 63% of MWRAs had a vaccination card present at the time of the survey [11]. Interestingly, only 24% of mothers reported having one in PSLM 2018–19, suggesting a need emphasize asking for one during training for PSLM surveys [12]. It would also be prudent to allow lessons from the in-depth questioning done for immunization in the PDHS and replicate it for the PSLM, as it was done for family planning. Internationally, researchers compare Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS) to assess health indicators, often identifying discrepancies in postnatal care estimates. MICS provides a more detailed postnatal care module, distinguishing immediate and later care, whereas DHS uses a blended module that does not systematically differentiate these phases [25]. Similarly, discrepancies in immunization estimates may arise because PDHS includes more detailed and specific immunization questions, potentially aiding respondent recall and improving data accuracy.
An additional factor that may contribute to the difference between the vaccination rates between the two surveys is how responses categories for this question are defined in each survey. In PSLM, both provincial and district level, the response category is “Yes, through the polio campaign”, while PDHS does not have this category. Table 4 shows that removing the “Yes, through the polio campaign” option from the “all-age appropriate vaccinations” narrows the difference from 23.5% to 20.2%. Thus, the adjustment may have a minor contribution to the difference between the two surveys regarding immunization. Similarly, removing the ‘yes, through the campaign’ category slightly reduces the difference for Polio and Measles indicators.
The one-year difference between the timing of the surveys may also have accounted for some of the observed differences in the vaccine administration estimates. PDHS took place in 2017–2018, while PSLM provincial surveys took place in 2018–19. For example, in response to a measles epidemic during 2017–2018, a nationwide Supplementary Immunization Activity (SIA) was conducted in October 2018, during which 37.1 million children aged under 5 were vaccinated. An independent post–SIA coverage survey estimated that SIA coverage of 93.3% [10]. The SIA may have also contributed to the difference between the Measles-Containing Vaccine 1 (MCV1) coverage rates, as it occurred between the two surveys. PDHS 2017–18 reported a 73.5% MCV1 coverage, whereas PSLM 2018–19 reported a 79.1% coverage. Similarly, there is a difference in Measles-Containing Vaccine 2 (MCV2) coverage, with PDHS 2018–19 reporting 66.75% coverage and PSLM 2018–19 reporting 79.7% coverage.
5. Conclusion
Our paper describes a high-level analysis with some suggestions of using available surveys to inform questions of national policy. We show complementarities, similarities in data collected and in quality, as well as some critical differences. PSLM, conducted biennially, with a larger sample size, and includes comprehensive economic details including income, expenditure, transactions, and assets. It is conducted regularly by the Pakistan Bureau of Statistics with government funds. PDHS is more in depth about health, with a fewer other subjects and smaller sample size, and is conducted ad hoc by the federal health ministry (the Ministry of National Health Services, Regulations and Coordination, through its National Institute of Population Studies), with mostly donor funds. There is a need to rationalize how the two surveys are conducted and how their data are used. A key ask has always been the ability inform about indicators at the district level. Only PSLM district survey does so, but only for a few indicators. It may be possible to add some indicators to it and then use data from either PDHS or PSLM provincial estimation techniques to impute other data to estimate family planning use which is not measured by the district PSLM survey. An alternative may be to incorporate all questions of PDHS into the PSLM provincial version, perhaps at lower frequency, for example around every 6 years. Understanding these similarities and limitations allows policy makers to develop confidence in these surveys and incorporate their evidence into their decisions.
Supporting information
S1 Data. The table below shows the comparison of questions asked in both PDHS PSLM surveys.
https://doi.org/10.1371/journal.pone.0320044.s002
(DOCX)
References
- 1.
Global Sustainable Development Report 2015 edition. New York; 2015.
- 2. Ndzabandzaba C. Brief for GSDR 2015: data sharing for sustainable development in less developed and developing countries. GSDR 2015. 2015.
- 3. Greenwell F, Salentine S. Health information system strengthening: standards and best practices for data sources. 2018.
- 4.
Portnoy B, Balluz L. Importance of comparable state and local health survey data for research to improve population health. American Public Health Association (APHA) 139TH Annual Meeting and Exposition; 2011.
- 5. Berti PR, Nardocci M, Tran MH, Batal M, Brodmann R, Greliche N, et al. Using DHS and MICS data to complement or replace NGO baseline health data: an exploratory study. F1000Res. 2021;10:69. pmid:34123367
- 6. Babazadeh S, Thanel K, Garfinkel D, Riley C, Bertrand J, Shaw B. FPwatch: facility-based survey data for family planning market analysis in five FP2020-focus countries. Stud Fam Plann. 2018;49(4):385–95. pmid:30451302
- 7. Anoke SC, Mwai P, Jeffery C, Valadez JJ, Pagano M. Comparing two survey methods of measuring health-related indicators: lot quality assurance sampling and demographic health surveys. Trop Med Int Health. 2015;20(12):1756–70. pmid:26425920
- 8.
Malanick C, Pebley A, editor. Data priorities for population and health in developing countries: summary of a workshop. Washington (DC): National Academies Press (US); 1996.
- 9.
Hox J, Boeije H. Data collection, primary vs. secondary. In: Encyclopedia of Social Measurement. 2005. p. 593–9.
- 10. Mere M, Goodson J, Chandio A, Rana M, Hasan Q, Teleb N. Progress toward measles elimination — Pakistan, 2000–2018. MMWR Morb Mortal Wkly Rep. 2019;68(10):505–10.
- 11.
Pakistan Demographic and Health Survey 2017-18. Islamabad, Pakistan, and Rockville, Maryland, USA; 2019.
- 12.
Pakistan Social And Living Standards Measurement 2018-19 National/Provincial (Social Report). 2020.
- 13.
Key finding report Pakistan social and living standards measurement (PSLM) survey: District level 2019-20. 2021.
- 14.
Pakistan Social and Living Measurement (PSLM) 2013-14 Provincial Report. 2015.
- 15.
Mohadjer L, Choudhry G. Adjusting for missing data in low-income surveys. Studies of welfare populations: data collection and research issues. 2002. p. 129–56.
- 16. Taub TW. Computation of two-tailed Fisher’s test. Journal of Quality Technology. 1979;11(1):44–7.
- 17. Newcombe RG. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med. 1998;17(8):857–72. pmid:9595616
- 18.
Tourangeau R, Rips L, Rasinski K. The psychology of survey response. Cambridge University Press; 2000.
- 19. Ramakrishnan R, Rao TV, Sundaramoorthy L, Joshua V. Magnitude of recall bias in the estimation of immunization coverage and its determinants. Indian Pediatr. 1999;36(9):881–5. pmid:10744865
- 20. Habib MA, Soofi SB, Ali N, Hussain I, Tabassum F, Suhag Z, et al. Knowledge and perceptions of polio and polio immunization in polio high-risk areas of Pakistan. J Public Health Policy. 2017;38(1):16–36. pmid:28077864
- 21. Dansereau E, Brown D, Stashko L, Danovaro-Holliday MC. A systematic review of the agreement of recall, home-based records, facility records, BCG scar, and serology for ascertaining vaccination status in low and middle-income countries. Gates Open Res. 2020;3:923. pmid:32270134
- 22. Faber J, Fonseca LM. How sample size influences research outcomes. Dental Press J Orthod. 2014;19(4):27–9. pmid:25279518
- 23. Khan Z. Immunisation and infant mortality in Pakistan. Pak Dev Rev. 1993;32(4 Pt 2):1117–23. pmid:12346810
- 24. Zaidi SMA, Khowaja S, Kumar Dharma V, Khan AJ, Chandir S. Coverage, timeliness, and determinants of immunization completion in Pakistan: evidence from the Demographic and Health Survey (2006-07). Hum Vaccin Immunother. 2014;10(6):1712–20. pmid:24784118
- 25. Amouzou A, Mehra V, Carvajal-Aguirre L, Khan SM, Sitrin D, Vaz LM. Measuring postnatal care contacts for mothers and newborns: An analysis of data from the MICS and DHS surveys. J Glob Health. 2017;7(2):020502. pmid:29423179