Introduction

An increasing awareness of the effects that cultural settings have on public health research and practice has led to a growing interest in understanding whether sociodemographic characteristics of interviewers affect the accuracy of survey data [1]. Considerable debate focuses on whether matching interviewer and respondent characteristics improves the accuracy of interview data [2, 3]. This concern is particularly relevant in household surveys that obtain self-reported information about sensitive topics through face-to-face interviews or in surveys that target respondents from marginalized populations [1, 4, 5].

The principal concern is whether observable characteristics of an interviewer (e.g., race, gender, age) or their perceived social status can bias responses to survey questions [1]. The literature considers three types of interviewer effects in survey research: non-participation (refusal to participate), nonresponse (refusal to answer a question) and response quality (accuracy of the response) [3, 4]. Methodological research carried out in high-income countries has focused on item nonresponse and strategies to control response bias using improved survey design and data analysis techniques [6]. Most studies of nonresponse consider interviewer or respondent characteristics alone. Few studies investigate the rapport between interviewer and respondent [7], and fewer still concern research in low resource settings [1, 6].

While it may seem logical that matching interviewer and respondent should reduce social distance and the potential for response error, evidence of interviewer effects across studies on response rates and response quality are inconsistent [1, 2, 5–9]. Several methodological studies show that interviewer effects vary by country and geographic area; however, little is known how cultural norms or social context influence the interviewer-respondent rapport and lead to measurable interviewer effects [1, 3, 4, 10, 11]. Few qualitative studies have explored respondents’ perceptions of the interview process, in particular the acceptability of an interviewer’s gender or how social norms influence female respondent behaviour and survey responses [12]. Such research is particularly lacking for traditional, hierarchical societies such as India, where gender and social norms often govern women’s decision making [13–15]. While some studies in India have investigated interviewer effects, few have considered the influence of gender in household surveys [16]. A study in Nepal revealed a pattern of under-reporting of selected family planning behaviours gathered by male interviewers in traditional communities [13]. However, there is limited available evidence from India on the effect of an interviewer’s gender on responses to household surveys assessing knowledge, attitudes and practices of mothers regarding maternal, newborn and child health (MNCH) services. Our research helps to fill this gap.

In September 2016, two identical MNCH household surveys were carried out by two different organisations in two rural districts of Bihar, India: Gopalganj which is located in the north of Bihar and Aurangabad in the south. The first survey was part of an NGO’s regular programme monitoring; the second one was a separate Third Party Monitoring of the NGO’s activities and the quality of its survey which was requested by the donor [17]. These were the sole objectives of the second survey. In addition to being located in the same districts at similar times, the surveys used identical questionnaires and similar sampling techniques. Respondents for both surveys were mothers of children aged 0–59 months. Rather than producing similar results, the two surveys displayed significant differences in over one-third of 156 comparisons of the indicators which assessed antenatal care (ANC) and birth preparedness, childhood nutrition, child immunization, frontline worker (FLW) support to mothers for breastfeeding, and family planning use [18].

The statistical differences between the two surveys are documented elsewhere [18] and reported among our results. While otherwise identical, the two surveys had one important difference that was not heretofore investigated, namely, the gender of interviewers differed between the two data collection teams. One survey team used all-male data collectors supported by male supervisors while the other had a team of all female data collectors supported by a mix of male and female supervisors. This situation provided an ideal opportunity to explore sociocultural norms regarding participation in household surveys in a rural setting in India and the influence of gender roles on survey responses, including question avoidance and accuracy.

We conducted a qualitative study from December 2017–July 2018 in Aurangabad and Gopalganj districts in Bihar, the same two districts where the twin household surveys had been implemented in 2016. Our objective was to explore the effects of interviewer gender on the interviewing process in the two districts where the quantitative surveys had been carried out. Both are primarily rural districts with a mix of upper caste and lower caste Hindu residents, and minority Muslim communities. Located in northeast India, Bihar is one of the country’s poorest and most densely populated states (N = 110million). It has some of the weakest maternal and child health and nutrition indicators in India [12, 19]. Bihar is a highly patriarchal traditional setting, characterised by low female literacy and autonomy, limited female agency in household decisions, poor spousal communication and low access to social support for married women [20]. Restrictive social norms perpetuate a chronic pattern of unequal gender roles and behaviours. Gender norms support male authoritarian roles, marginalize women’s voice and influence, associate male honour with a woman’s purity and condone male violence toward women [21]. Gender-based violence is pervasive in Bihar which has India’s highest rate of reported domestic violence [22].

Our principal research questions are: How do social norms (cf [23]) affect a mother’s willing participation in household surveys in two rural districts in Bihar, India? How do gender roles influence her ability to respond and the accuracy of her responses to survey questions on MNCH topics? We initially explore the perceived acceptability of male and female interviewers for household surveys on MNCH topics that target mothers of young children. Then we explore how prevailing gender roles and social constraints may influence their survey participation, nonresponse and response quality. Finally, we discuss implications of our findings for future household surveys. We identify survey content that is susceptible to interviewer gender effects and may influence data accuracy and we interpret the discrepancy in the results of the two 2016 surveys.

Methods

We collected qualitative data using focus group discussions (FGD) and in-depth individual interviews (IDI) with a range of participants to explore perceptions, beliefs and experiences related to gender and social norms in two districts in Bihar. FGDs were conducted with mothers and significant family members, including husbands, mothers-in-law and grandparents. In a few cases (in particular, with upper caste mothers), participants refused to join the FGD but agreed to an IDI instead. Other IDI participants included frontline workers (FLW) who are Accredited Social Health Activists (ASHA) and Anganwadi Workers (AWW). We also conducted in-depth telephone interviews with supervisors and enumerators involved in the earlier household surveys in the same two districts.

Using semi-structured topic guides, we explored how sociocultural norms and processes affect a mother’s participation in household surveys and her ability to respond accurately to survey questions asked by an interviewer of the same or opposite gender. Interview questions focused most intensely on family planning, antenatal care, birth planning, childhood nutrition, and childhood immunizations as these were the indicators with substantial differences between the two household surveys carried out previously in Bihar. We also asked questions to understand reasons why the two surveys agreed on other indicators. The data collection tools, which were developed for this study, can be found in the S1 File. They include Focus Group Discussion Guides for Mothers of Children < 5 Years and Household members (husbands, grandparents, other in-laws living in households with mothers of children < 5 years); and Individual Interview Guides for ASHA/AWW and Interviewers/Supervisors from the prior probability surveys in Bihar (see S1 File).

Sampling

We purposefully selected four villages (two each in Aurangabad and Gopalganj districts) using the sampling frame from the 2016 household surveys comprising a list of all villages and the approximate population sizes. Subsequently the field team purposefully selected 45 mothers for five FGDs. The inclusion criteria were residence in Bihar for at least 6-months and having a child under five years old. A second purposive sample of 52 household members was selected for a separate set of five FGDs. These individuals were adult family members living in the same household as a mother selected for interview. Male respondents included husbands or other male family members (e.g., brother-in-law, father-in-law, grandfather). Female respondents included family members such as mother-in-law, sister-in-law or grandmother. FDGs for male and female participants were run separately. Eleven individuals (10 mothers, 1 husband) selected for an FDG declined to participate but agreed to an IDI instead. Eleven frontline workers (FLW) working in the two study districts were recruited for IDIs using convenience sampling (based on availability at the time the research team was present in the study district). Lastly, four enumerators and seven supervisors (male and female) from past household surveys in Bihar were purposefully selected for IDIs carried out by phone. All eligible study participants were conversant in Hindi or English and were over 18 years of age (Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Sample size by type of participant and interview method.

https://doi.org/10.1371/journal.pone.0252120.t001

So that our sample reflected the diversity in the population, the field team developed community maps in each sampled village in both districts with the help of the local AWW. These maps revealed the geographical distribution of different caste groups and the location of marginal communities in each village. The field team used the maps and existing household lists to purposively select respondents from different castes, social groups and locations within the villages to ensure the sample accommodated multiple voices from different economic groups. Selection intended to include marginalized people whose opinions often are not sought, such as the poorest, young mothers, scheduled castes, and minority religions. Economic groups tended to live in the same enclaves within villages. Using community maps to guide the purposive sampling ensured that mothers from the poorest and wealthier groups were included. However, we did not ask for within-caste distinctions so as to maintain the integrity of the selected focus groups. Our choices were verified by the AWW.

Results

Differences in results of two household surveys

Interviewer gender effects may help explain some of the differences in results that surfaced when comparing the two household surveys carried out in Bihar in 2016. The surveys were implemented in the same two districts during similar timeframes using identical questionnaires and similar probability sampling designs. We compared district coverage estimates for 54 indicators in both survey datasets. Some indicators were measured in multiple child age groups resulting in 156 comparisons of indicators; 58 comparisons (37.2%) differed significantly at the 0.05 level. We use the data collected by female interviewers as the reference (see Table 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Significantly different results from data collected by male interviewers compared with data collected by female interviewers in Bihar, India.

https://doi.org/10.1371/journal.pone.0252120.t002

The differences between the two datasets were mixed, and go in both directions, indicating a lack of a systematic bias. In 39 (25%) of the comparisons, coverage data collected by male interviewers was significantly lower than those of female interviewers while in 19 (12%) of them, the male interviewer displayed significantly higher results [18]. However, the discordance between the two datasets was not limited to gender sensitive questions as we expected. We anticipated that a team of male interviewers would exhibit underestimations of gender-sensitive topics and no discordance on child-focused topics, relative to the data collected by females. As expected, we found significant underestimation of family planning coverage (14.3%) and reported FLW home visits during pregnancy and perinatal period (38.9%). Surprisingly, we also found significant underestimation of the two child-focused topics (childhood nutrition (44.4%), immunization (31.3%) and significant overestimation of coverage of delivery/newborn care services (11.8%). Also, the coverage results for ANC/birth preparedness were similarly overestimated (23.7%) and underestimated (23.7%).

Discussion

Three overarching but interconnected findings in this study reflect deeply internalized social and gender norms that affect a mother’s ability to discuss gender-sensitive matters with a male interviewer in our study districts in Bihar: (1) mothers have limited agency to communicate with men about personal health issues, (2) gendered communication is influenced by restrictions of caste and religion, and (3) interviewer gender effects are likely associated with the survey topic.

Firstly, most mothers in our study exercised limited agency to communicate with any man (within the family or outside community members) about their personal health issues, even in the context of a household survey. They also lacked support within their families for crossing gender lines to discuss sensitive health issues. Regardless of age, caste and educational level, mothers were unable to talk to other men (aside from their husband) about female-specific topics. Other studies in India have shown that female modesty engrained in societal expectations linked to gender and social status make many women, especially younger women, reluctant to talk about any reproductive health matters with men, except professional health workers [16, 31, 32].

In traditional Indian settings, family and kinship structures usually determine what is considered to be acceptable and appropriate behaviour for most women. “Respectful avoidance” [16, 31, 32] is a prevailing social norm related to a woman’s social status. It is understood that a married woman should avoid speaking to any male relative who is the same age or older than her husband. In contrast, most women are able to have informal relationships with their husbands’ younger brothers. Thus, avoidance is considered appropriate behaviour for interactions with people of higher status which, in this case, means elder men in the family such as an elder brother-in-law and father-in-law. But women may engage freely with individuals of equal or lower status, such as a younger brother-in-law or other women in the family [16].

In our study area, communicating with a man may be akin to physical intimacy which is prohibited and unacceptable for a married woman; this reaction was also noted in an earlier study in southern India [32]. Given this context, if a mother is seen talking to a male interviewer, the scene can be easily misconstrued by onlookers leading to negative and even harmful consequences for the her. Most mothers in our study area preferred to avoid this situation. Mothers overwhelmingly preferred to discuss their personal health issues with a female interviewer as they perceived fewer risks if observed sharing information with an outside woman. These findings are consistent with other studies that discuss the restrictive gender roles in India regarding intra and extra-familial communication on health issues [31, 32].

Secondly, our findings draw attention to the ways in which gender, caste, religion and other social factors intersect to shape a mother’s everyday experiences with restrictions on communication and sharing information in our study districts. Upper and lower caste and Muslim mothers in our study face a different set of challenges regarding communication with a male interviewer. Of note is the difficulty in gaining entry into Hindu upper caste and Muslim households to interview mothers for survey research, and how this differs from access to mothers from the Hindu lower caste. We found that all mothers must be cautious in their interactions with men outside their family; however, this is especially salient for upper caste mothers. Social norms in India are more binding among upper caste women and failure to uphold them carries risk [33, 34]. Family honour is tightly tied to a woman’s behaviour which is especially true for the upper castes. Preserving the husband’s family honour as well as the outward appearance of respectability can significantly impact on a woman’s status and even her physical safety [31, 32]. It is not surprising that upper caste mothers in our study generally opposed any interaction with male enumerators whenever possible. Upper caste and religious constraints on a mother’s participation in a household survey, including strict observation of seclusion norms, could lead to selection bias in survey research through her self-exclusion. Fear of disapproval and opposition from peers and family may suppress the participation of upper caste and Muslim mothers, leading to a persistent lack of information about their health status and health care needs in survey data.

In contrast, lower caste mothers were more open to participating in a survey administered by a male interviewer. These women generally have more permissive families and greater mobility due to working as labourers [20]. While they may perceive fewer restrictions than upper caste mothers with regard to communicating with men outside of traditional family circles, they can face severe social hostility from upper caste communities for communicating across gender lines. Future studies may consider exploring the intersecting influences of caste, gender and other social factors (e.g., age and class) for a more nuanced understanding of interviewer effects.

Thirdly, we found certain interview topics were more likely to be susceptible to interviewer gender effects when included in a household survey, including questions about pregnancy or ANC, reproductive choices, breastfeeding, sexual behaviour and STIs. Mothers are willing to respond to these types of questions but only to a female enumerator. The only man with whom they can safely discuss these topics is their husband. In contrast, topics related to children’s health have no such restrictions. Mothers were willing and able to share health information about their children with interviewers of either gender. They generally experienced favourable social support within their family and community for crossing gender lines to discuss children’s health issues. Concern for their children’s wellbeing outweighs the potential downsides related to perceived disrespect and shame from being seen talking to an outside man. This may be less pronounced for upper caste mothers who prefer to avoid male interviewers altogether, regardless of the topic.

These findings from our qualitative research help us to understand the different results produced by the two probability household surveys. It is plausible that in these surveys, both under- and overestimated coverage reflect mothers’ attempts at survey avoidance due to the pressures of gender norms and social constraints. Low coverage for an indicator is usually interpreted to mean poor uptake of the particular service by the target population. However, lower coverage in this case may be due to the mother’s attempt to avoid a question or to her refusal to respond to the male interviewer in the NGO survey. This may be due to mothers’ unwillingness and inability (low agency) to respond accurately to questions posed by a male interviewer. In a survey, a refusal to answer a question typically results in a negative response, which reduces the coverage proportion for an indicator. Likewise, skipped questions often generate a cascading series of negative responses which again can artificially reduce the indicator value for that particular service. Alternatively, non-sampling error due to interviewer gender may reflect a situation where husbands (or other male family members) respond on behalf of their wives and offer inaccurate information in either direction (positive or negative responses to questions).

Nevertheless, interviewer gender effects do not always lead to underreporting due to negative responses. Avoidance can also occur when a mother responds YES in an attempt to end the survey quickly, which leads to higher counts resulting in erroneously overestimated coverage estimates. While we cannot know a priori which way the effects will go, the clear discordance of results in Table 2 suggest survey data in Aurangabad and Gopalganj districts may be unreliable when collected by a male enumerator for the reasons already outlined. Unreliable data due to effects of the interviewer-respondent relationship is a pattern seen in other methodological work investigating the role of the interviewer in household surveys [6].

Similar mixed findings have been reported in studies in other non-Western settings. For example, a case study of interviewers for a family planning intervention in Ghana showed differential responses given by women and men in surveys due an interviewer’s gender; responses concerned knowledge and use of contraceptives and the frequency of sexual activity [35]. A Nigerian study in Muslim neighbourhoods noted that a male member of the family was always present when the interviewer was a man which inhibited a woman’s answers to questions about family planning. Another Nigerian study also revealed discordant responses to questions on knowledge and use of contraception with male versus female interviewers; errors were bi-directional [36]. A study in Ghana cautioned against predicting the direction of response bias due to interviewer-gender effects [37]. That study compared survey responses to male and female interviewers and found differences in responses to knowledge and attitude questions regarding HIV/AIDS; however, there were few discrepancies in sexual behaviour reported to male enumerators. In contrast, a study in Zambia found, female respondents were more likely to report sexual behaviours to female interviewers than to males [11].

Our findings are consistent with these examples and underscore the likelihood that interviewer gender can introduce non-sampling error, and bi-directional error in household surveys with MNCH topics in traditional settings such as Aurangabad and Gopalganj. When respondents refuse to answer or they answer but respond incorrectly, the survey conclusions will be erroneous. Health policy decisions based on unreliable or inaccurate data may be misguided as a result.

Conclusions

This study explores gender-specific constraints to participation in household surveys in Aurangabad and Gopalganj districts and how these constraints can affect the interpretation and accuracy of survey findings on MNCH indicators. Our findings highlight the influence of the interviewer-respondent relationship in the traditional and hierarchical setting of northern India with a culture of strict observance of social restrictions linked to gender and caste. Researchers need to be aware of how the interviewer-respondent dyad influences the capturing of survey data and biases that can result due to social distances.

In this case, social norms and gender biases in our study districts may have affected whether standard questions used in household surveys accurately measured coverage of topics ranging from childhood immunization to gender-sensitive services such as family planning, ANC and delivery care. Interviewer gender is a likely source of non-sampling error in household surveys when a male interviewer gathers data from a female respondent about gender sensitive issue. This factor may be particularly true in a traditional setting such as Aurangabad and Gopalganj.

Given the challenges of capturing accurate data through face-to-face surveys, research teams can take preventative measures to mitigate the potential impact of interviewer characteristics, in particular when designing protocols and questionnaires for maternal and reproductive health studies and when selecting and training the data collection team. Our findings highlight the importance of training enumerators to establish trust and confidence and to enquire when obtaining “informed consent” if the female or male respondent views specific categories of survey questions as culturally sensitive. We also suggest screening interviewer candidates for gender bias prior to hiring them for a household survey. For example, this can be done during the recruitment process by asking male and female interviewer candidates about their own preferences for an interviewer’s gender, if they are uncomfortable interviewing respondents of either gender and if they have any particular beliefs or preconceptions about gender roles. While this type of interviewer screening may be an afterthought in some analyses, we suggest it as a standard operating procedure. Similarly, using qualitative methods as part of the survey design stage will help reveal sociocultural norms of the study setting that may help avoid biased results in the larger survey.

Social distance between interviewer and respondent can be minimized in household surveys on MNCH topics carried out in a traditional Indian context like rural Bihar. Practical actions include matching interviewer and respondent characteristics, especially gender and caste/religion, and when not possible, randomizing interviewer characteristics across respondents. This advisory may vary outside of South Asia. Female interviewers should be considered an optimal choice for areas such as Aurangabad and Gopalganj districts to facilitate access to mothers, reduce discomfort during the interview process and obtain better quality data with more accurate and consistent responses to sensitive topics. MNCH surveys in settings such as our study districts that consistently use all male interviewers, when gender effects are possible, may result in bias; when repeated across studies this bias can alter our understanding of public health issues and affect policy decisions. Misguided policy and programme responses can occur as a consequence of biased survey data. Future research may consider applying an intersectionality framework to analyse the influence of gender, caste, religion, age and other social factors on survey research in rural India. Intersectionality can help us to understand how multiple factors can overlap and combine to result in different types of restrictions or discrimination experienced by female respondents.

This research provides context for the unexplained differences in the quantitative results from the twin surveys carried out in Aurangabad and Gopalganj in 2016. While we add to the methodological research on the interviewer-respondent dyad in India, our findings have implications for elsewhere. These results raise serious issues for protocol design for household survey research in any traditional setting where women’s autonomy is limited and suggest the need for additional research systematising the effect of gender when measuring population health. Systematic examination of the potential for interviewer-gender effects and a better understanding of the sociocultural norms are needed in different contexts where surveys take place.

Supporting information

Acknowledgments

We gratefully acknowledge the essential roles of Baburam Devkota, Hemant Das, Alok Prahdan and Sanjay Biswa for their careful field work supporting the LQAS survey and managing the study team in Bihar.