https://orcid.org/0000-0002-7037-6658
Figures
Abstract
Introduction
Although interactive voice response (IVR) is a promising mobile phone survey (MPS) method for public health data collection in low- and middle-income countries (LMICs), participation rates for this method remain lower than traditional methods. This study tested whether using different introductory messages increases the participation rates of IVR surveys in two LMICs, Bangladesh and Uganda.
Methods
We conducted two randomized, controlled micro-trials using fully-automated random digit dialing to test the impact of (1) the gender of the speaker recording the survey (i.e., survey voice); and (2) the valence of the invitation to participate in the survey (i.e., survey introduction) on response and cooperation rates. Participants indicated their consent by using the keypad of cellphones. Four study arms were compared: (1) male and informational (MI); (2) female and information (FI); (3) male and motivational (MM); and (4) female and motivational (FM).
Results
Bangladesh and Uganda had 1705 and 1732 complete surveys, respectively. In both countries, a majority of the respondents were males, young adults (i.e., 18-29-year-olds), urban residents, and had O-level/above education level.
In Bangladesh, the contact rate was higher in FI (48.9%), MM (50.0%), and FM (55.2%) groups than in MI (43.0%); the response rate was higher in FI (32.3%) and FM (33.1%) but not in MM (27.2%) and MI (27.1%). Some differences in cooperation and refusal rates were also observed. In Uganda, MM (65.4%) and FM (67.9%) had higher contact rates than MI (60.8%). The response rate was only higher in MI (52.5%) compared to MI (45.9%). Refusal and cooperation rates were similar.
In Bangladesh, after pooling by introductions, female arms had higher contact (52.1% vs 46.5%), response (32.7% vs 27.1%), and cooperation (47.8% vs 40.4%) rates than male arms. Pooling by gender showed higher contact (52.3% vs 45.6%) and refusal (22.5% vs 16.3%) rates but lower cooperation rate (40.0% vs 48.2%) in motivational arms than informational arms. In Uganda, pooling intros did not show any difference in survey rates by gender; however, pooling by intros showed higher contact (66.5% vs 61.5%) and response (50.0% vs 45.2%) rates in motivational arms than informational arms.
Conclusion
Overall, we found higher survey rates among female voice and motivational introduction arms compared to male voice and informational introduction arm in Bangladesh. However, Uganda had higher rates for motivational intro arms only compared to informational arms. Gender and valence must be considered for successful IVR surveys.
Trial registration
Name of the registry: ClinicalTrials.gov. Trial registration number: NCT03772431. Date of registration: 12/11/2018, Retrospectively Registered. URL of trial registry record: https://clinicaltrials.gov/ct2/show/NCT03772431?term=03772431&cond=Non-Communicable+Disease&draw=2&rank=1. Protocol Availability: https://www.researchprotocols.org/2017/5/e81.
Citation: Labrique A, Nagarajan M, Kibria GMA, Vecino-Ortiz A, Pariyo GW, Ali J, et al. (2023) Improving success of non-communicable diseases mobile phone surveys: Results of two randomized trials testing interviewer gender and message valence in Bangladesh and Uganda. PLoS ONE 18(5): e0285155. https://doi.org/10.1371/journal.pone.0285155
Editor: Jake Michael Pry, University of California Davis School of Medicine, UNITED STATES
Received: April 21, 2022; Accepted: April 17, 2023; Published: May 24, 2023
Copyright: © 2023 Labrique et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are available at the OPENICPSR repository: https://www.openicpsr.org/openicpsr/project/183511/version/V1/view.
Funding: This study was made possible with the support of Bloomberg Philanthropies (https://www.bloomberg.org/) and the people of Australia through the Department of Foreign Affairs and Trade (https://dfat.gov.au/pages/default.aspx) through award number 119668. The contents are the responsibility of the authors and do not necessarily reflect the views of Bloomberg Philanthropies or the Government of Australia. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Over the past several decades, survey research has been challenged by the exponential increase in the volume of digital traffic vying for attention [1]. Whether internet, email, or phone-based–surveys struggle to capture and hold a putative respondent for even a few minutes, fighting against a supersaturation of digital requests and so-called “spam” commercial messages and calls [2,3].
The use of high-volume commercial robocalling around the world has led to a distrust of and non-response to random calls from unknown numbers and proliferation of spam blockers [2,3]. Many commercial and clinical interactions are followed by a set of questions or require a post-experience quality assessment, rapidly giving rise to “survey exhaustion” [4]. From the intentional masking of phone numbers to seem authentic to the use of catchy images or messages, a plethora of strategies, both genuine and nefarious, have been attempted in efforts to increase engagement and participation [5,6].
Despite these obstacles, interactive voice response (IVR) surveys remain popular and important tools for public health, relying on automated calls to ask respondents to press keypad numbers in answer to questions about their risk factors, health, and environment [7,8]. National public health surveys such as the Behavioral Risk Factor Surveillance System (BRFSS) in the United States compete with the high number of daily spam calls, thus struggling to meet their sample sizes [9,10]. Data from the Pew Research Center, a research group that conducts the General Social Survey in the U.S., shows a steady decline in phone survey response rates since the late 1990s, with response rates previously as high as 36% dropping to less than 10% in 2016 [11].
While many low- and middle-income countries (LMICs) have “leapfrogged” the fixed phone line and directly entered the cellular era [12], comparable challenges of low survey response rates confront researchers conducting mobile phone surveys in LMIC settings. Despite this jump directly to mobile phones, the challenges of unwanted calls and messages remain [13]. A 2019 study of the global burden of these “spam” messages to mobile phones found an average of 119 unwanted text messages per month going to users in Ethiopia, to 27 and 61 per month going to users in Bangladesh and India, respectively [14].
Drawing on intrinsic and extrinsic motivators to increase responses to surveys is a well-researched area [15]. One approach is to position research participation as an important societal responsibility, another is to motivate survey participation through financial incentives or rewards [16–18]. Cash incentives have been shown to be successful in reducing non-response in mobile phone surveys (MPS) as well as attrition rates in medication adherence [17,19–22]. A meta-analysis quantifying the dose-response relationship between incentives and response rates in household surveys (whether via mail, phone, or in-person) found a nonlinear relationship between incentives and an increased response rate [23]. Mercer et al. reported that “…respondents value nonmonetary incentives differently than monetary incentives. Gifts may have other characteristics beyond their perceived cash value that determine their worth” [22].
In the broader literature on postal and landline-based telephone surveys, innovations range from the use of emotional motivation to the selection of culturally appropriate voices [21,23]. Some of the evidence for the effectiveness of content that draws on emotional drivers to capture and retain respondents is borrowed from advertising research and behavioral interventions [24]. Programs such as Swacch Bharat (translated as “Clean India”) feature nostalgia-inducing advertising by celebrities to promote civic engagement; this approach leverages patriotic/group pressure to trigger action. This has not, however, been well-tested in phone-based surveys [25].
The perceived gender of the recorded voice may also play a role in capturing initial respondents as well as in their willingness to answer sensitive questions. Further, the relative effectiveness of male versus female voices may vary by the gender of the respondent, context, and subject matter [26]. This effectiveness (or lack thereof) could have implications for the completeness of data and minimizing gender data gaps in mobile data collection efforts. Though differential effects of the interviewer’s voice and perceived gender, across respondent gender, have been reported in other studies [27,28], few studies investigated the combined impact of gender and type of introductory message on mobile phone survey performance, especially in LMICs. The ubiquitous growth of mobile phones in LMICs has been accompanied by the potential to use this platform for public health data collection, however, understanding the impact of introductory messages will be helpful to design and implement MPS in a shorter time and at a lower cost. The study aims to fill these gaps in knowledge and assess how different introductory message characteristics (i.e., gender of speaker recorded and valence of the introductory message) affect survey performance metrics of a non-communicable disease (NCD)-risk factor IVR questionnaire in Bangladesh and Uganda.
Methods
Study design
We conducted two randomized controlled micro-trials to test ways to improve the performance of IVR MPS in Bangladesh and Uganda. As indicated in Fig 1, participants were randomized to one of four arms with messages varying by male or female voice, and whether the invitation to participate in the study was paired with information or motivational messaging. Four arms were included in both countries: 1) male voice, informational introduction (MI); 2) female voice, informational introduction (FI); 3) male voice, motivational introduction (MM); and 4) female voice, motivational introduction (FM). We also used pooling, which generated two arms across introductory messages: 1) male voice (MV) and 2) female voice (FV); and survey voices: 1) informational intro (II) and 2) motivational intro (MI). The allocation ratio was 1:1:1:1.
Consolidated Standard of Report Trial Diagram of the Study Arms Enrollment in Bangladesh (1a) and Uganda (1b).
Randomization and masking
Study participants were sampled using the random digit dialing (RDD) method. Participants who received the call and connected to the IVR system were screened for age greater or equal to 18 years old to determine participation eligibility. Age-eligible participants were screened for willingness to participate in the study (answer survey questions), indicated by response choice on their mobile phones. Eligibility- and consent-screened participants were presented with the remainder of the IVR survey [29].
A fully automated pseudo-random number generator was used to produce random numbers of seven-digit length, prefixed with (randomly generated) mobile network operator-specific three-digit phone codes and the country codes for Bangladesh (880) and Uganda (256). Participants were randomized to their respective study arms after choosing their preferred survey language, and the randomization was through an automated process built into the IVR platform by the provider. Study participants could not be masked to their study group allocation due to the nature of the intervention, namely different genders of voices and different messages on the recordings played to the participants [30]. The data cleaning was done by researchers blinded to the allocation of study participants.
Procedure
The study procedures for Bangladesh and Uganda were comparable. Participants who picked up the phone call were asked to indicate language preference through a numeric response on the keypad. In Bangladesh, the languages available were Bangla and English; in Uganda, the languages available were Luganda, Luo, Runyakitara, and English. In both countries, the languages offered covered the majority of the population.
Survey questions were administered in the following order of broad categories: 1) language selection; 2) introduction to the survey, including the description of the requirements to be met in order to receive an incentive; 3) screening questions (age-eligibility, consent); 4) demographic questions; and 5) five NCD modules. The NCD modules are related questions grouped as they pertain to 1) tobacco use, 2) alcohol use, 3) diet, 4) physical activity, and 5) high blood pressure and diabetes markers. While the order of administration of different NCD modules was randomized to minimize bias due to drop-off or attrition mid-survey, skip patterns within modules were preserved by keeping the order of questions in each NCD module constant. After confirming the age, eligible participants were asked to provide consent by pressing number 1 button on mobile phone’s screen.
The questions in each module in the IVR survey were based on standardized questions from surveys such as the World Health Organization’s STEPwise surveillance [31] and Tobacco Questions for Surveys (TQS) [32]. They were chosen by experts in NCDs and compiled in an initial questionnaire. Cognitive understanding and usability were initially tested at Johns Hopkins University, with persons who identified as being from a low- or middle-income country setting [33]. A series of key informant interviews (KIIs), focus group discussions (FGDs), and user groups were conducted in each country, and feedback was used to refine and adapt the questionnaire and deployment to the country context [34]. Audio files of the questionnaire were tested at the country level prior to survey deployment to ensure that translations and recordings were comprehensible.
The IVR surveys were deployed between 08:00 AM and 08:00 PM local time with a single attempt made to contact each randomly generated number. Survey participants could choose to repeat questions through key presses on their mobile phones as they moved through the survey. Participants did not incur charges for the airtime taken to complete the survey and were informed of this; those who completed the IVR survey would receive a small airtime incentive delivered through their provider. The scripts of the motivational and informational intros are shown in S1 Table.
The two micro-trials generated 361,740 calls with 1,705 surveys completed in Bangladesh (Fig 1A), and 87,240 calls placed with 1,732 complete surveys in Uganda (Fig 1B). These trials ran between March and April of 2017 in Uganda and June and July of 2017 in Bangladesh. The number of participants in MI, FI, MM, and FF was, respectively, 426, 445, 417, and 417 in Bangladesh, and 385, 464, 477, and 403 in Uganda.
Outcomes
Survey outcomes (i.e., survey performance metrices) were categorized using standard definitions of disposition codes and survey rates from the American Association for Public Opinion Research (AAPOR) [35]. The definitions of the disposition codes as used in this study and equations for primary and secondary outcomes are listed in Table 1.
The primary outcomes of the study were the response rate and the cooperation rate. The response rate was the sum of complete and partial interviews divided by the sum of all possible eligible participants (i.e., complete and partial interviews, refusals, breakoffs, and the estimated proportion of age-eligible unknowns). The cooperation rate was the number of complete interviews divided by the sum of complete and partial interviews, refusals, and others. The secondary outcomes were the contact rate and refusal rate. The contact rate is the sum of complete and partial interviews, refusals, and others divided by the sum of all possible eligible participants; the refusal rate is the number of refusals over that same denominator (Table 1).
Statistical analysis
Sample sizes for Bangladesh and Uganda were calculated under the assumptions of a control arm cooperation rate of 30%, an alpha of 0.05, and a power of 80%. This calculation meant 376 completed survey responses were needed in each arm to detect a 10% absolute difference in cooperation rates between two study arms. We did not inflate sample sizes for multiple comparisons, as per the recommendation by Rothman [36].
Demographic characteristics of the participants with complete interviews and disposition codes of all phone calls were compared across the four study arms in both countries. Log-binomial regression was used to calculate the risk ratio (RR) and corresponding 95% confidence interval (CI) for contact, response, refusal, and cooperation rates, using the male voice and informational introduction as the reference group for the four study arms. Data were pooled by survey voice and survey introduction to compare the isolated effects of survey voice and survey introduction on contact, response, refusal, and cooperation rates (RR and 95% CI) in both countries. Sensitivity analyses were run comparing respondent demographics of complete and partial interviews, and using different definitions of the outcome rates, calculated without applying e for the unknown participants (Table 1). Analyses were conducted using Stata/SE (version 14.1; Stata Corp, College Station, TX, USA) [37]. An alpha of 0.05 was assumed for all tests of statistical significance.
Ethical approval and consent to participate
Ethical clearance was received from the institutional review boards of the institutions involved in the micro-trials, namely the Johns Hopkins University Bloomberg School of Public Health, Baltimore, Maryland, U.S.; Makerere University School of Public Health, Uganda; the Uganda National Council for Science and Technology, Uganda; and the Institute of Epidemiology Disease Control and Research, Bangladesh. Eligible study participants had to provide informed consent to participate using keypad of cell phones. After confirming the age by pressing mobile phone numbers, eligible participants were provided the consent disclosure statement. They were asked to provide consent by pressing the 1 button on the mobile phone.
Results
In Bangladesh, the demographic characteristics of complete interviews were balanced across the 4 arms (Table 2). A majority of the study participants were male (84.7%-87.8%) between the ages of 18–29 years (64.6% - 68.1%). The most common levels of education reported were between O-level (24.5% - 29.9%), and A-level (22.3% - 25.8%) equivalents. About half the respondents were from an urban location (47.7% - 56.6%), and almost all took the survey in Bangla (over 98%).
In Uganda, as in Bangladesh, the demographic characteristics were balanced across the four study arms (Table 2). More than two-thirds of the participants were males (74.2% - 77.3%) between the ages of 18–29 years (68.8% - 72.8%). The most common responses to highest level of education were primary-level (23.9% - 27.6%) and O-level (22.8% - 27.8%). Slightly more than half of the study population was in an urban location (51.7% - 58.3%). A majority of survey respondents elected to take the survey in Luganda (57.1% - 62.0%), followed by Runyakitara (14.5% - 21.6%), English (14.4% - 18.0%), and Luo (4.2% - 6.5%).
Table 3 examines the disposition codes of all calls placed across the four study arms. The estimated proportion of age-eligible unknowns, calculated using the known proportion methodology [38], was 79.0% (Bangladesh) and 90.7% (Uganda).
Primary outcomes
Table 4 exhibits the survey rates for completed interviews across the study arms in Bangladesh and Uganda. In Bangladesh, the cooperation rates (i.e., the number of complete interviews divided by the sum of complete and partial interviews, refusals, and others) for the four study arms were: MI (43.9%); FI (53·2%); MM (37·3%); and FM (43·1%). Compared to the MI arm, the cooperation rate was significantly higher for the FI arm (RR: 1·21, 95% CI: 1·10–1·33, p<0.001) and significantly lower for the MM arm (RR: 0·85, 95% CI: 0·76–0·94, p = 0.002), with no detectable difference for the FM arm (RR: 0·98, 95% CI: 0·89–1·09, p = 0.71) in Bangladesh. In Uganda, the cooperation rates for the four study arms were: MI (56·5%); FI (56·6%); MM (37·3%); and FM (61·6%). With the MI arm as the reference, there was no detectable difference in the cooperation rate for the FI arm (RR: 0·94, 95% CI: 0·86–1·02, p = 0.13), the MM arm (RR: 0·98, 95% CI: 0·86–1·02, p = 0.13), and the FM arm (RR: 1·03, 95% CI: 0·93–1·11, p = 0.6) in Uganda.
In Bangladesh, the response rates (i.e., number of complete interviews divided by the number of all possible eligible participants) of four study arms were: MI (27·1%); FI (32·3%); MM (27·2%); and FM (33·1%). With the MI arm as the reference, the response rate was significantly higher for the FI arm (RR: 1·19, 95% CI: 1·08–1·31, p<0.001) and the FM arm (RR: 1·22, 95% CI: 1·11–1·35, p<0.001), with no detectable difference for the MM arm (RR: 1·00, 95% CI: 0·91–1·10, p = 0.95) in Bangladesh. In Uganda, the response rates for the four study arms were: MI (45·9%); FI (44·6%); MM (48·1%); and FM (52·5%). With the MI arm as the reference, there was no detectable difference in the response rate for the FI arm (RR: 0·97, 95% CI: 0·89–1·06, p = 0.54) and the MM arm (RR: 1·05, 95% CI: 0·96–1·14, p = 0.28), and an increased response rate in the FM arm (RR: 1·15, 95% CI: 1·05–1·25, p = 0.003) in Uganda (Table 4).
Secondary outcomes
Contact rates (indicating the proportion of persons reached by the survey) were 43.0%, 48.9%, 50.0%, and 55.2%, respectively in MI, FI, MM, and FM arms in Bangladesh; there was a significant increase in the contact rate in the FI (RR: 1·14, 95% CI: 1·06–1·22, p<0.001), MM (RR: 1·16, 95% CI: 1·09–1·24, p<0.001), and FM (RR: 1·28, 95% CI: 1·20–1·37, p<0.001) arms in Bangladesh compared to the MI arm. In Uganda, the Contact rates (indicating the proportion of persons reached by the survey) were 43.0%, 48.9%, 50.0%, and 55.2%, respectively in MI, FI, MM, and FM arms in Bangladesh; there was a significant increase in the contact rate in the FI (RR: 1·14, 95% CI: 1·06–1·22, p<0.001), MM (RR: 1·16, 95% CI: 1·09–1·24, p<0.001), and FM (RR: 1·28, 95% CI: 1·20–1·37, p<0.001) arms in Bangladesh compared to the MI arm. In Uganda, the contact rates across the four arms were 60·8%, 62·1%, 65·4%, and 67·9%, respectively in MI, FI, MM, and FM arms; compared to the MI arm, there was no difference in the contact rate in the FI arm (RR: 1·02, 95% CI: 0·96–1·09, p = 0.55), and an increase in the MM (RR: 1·07, 95% CI: 1·01–1·14, p = 0.024) and FM (RR: 1·12, 95% CI: 1·05–1·19, p = 0.001) arms (Table 4).
The refusal rates (indicating the proportion of breakoffs and refusals among those eligible to participate in the survey) were 16.0%, 16.6%, 22.9%, and 22.1%, respectively, across the four study arms (i.e., MI, FI, MM, and FM) in Bangladesh. With the MI arm as reference, there was no difference in the contact rate in the FI arm (RR: 1·04, 95% CI: 0·90–1·20, p<0.001), and an increase in the MM (RR: 1·43, 95% CI: 1·27–1·62, p<0.001) and FM arms (RR: 1·38, 95% CI: 1·21–1·57, p<0.00`). In Uganda, the refusal rates were 15.0%, 17·5%, 17·3%, and 15·4% in MI, FI, MM, and FM arms, respectively. With the MI arm as reference, there was no difference in the refusal rate in the FI arm (RR: 1·16, 95% CI: 0·97–1·40, p = 0.11), the MM arm (RR: 1·15, 95% CI: 0·96–1·39, p = 0.13), or the FM arm (RR: 1·02, 95% CI: 0·83–1·26, p = 0.83) (Table 4).
Pooled analysis
Table 5 evaluates group samples to look for the effects of survey voice introduction in both countries. Using the pooled sample, in Bangladesh, the cooperation rates were: MV (40·4%), FV (47·8%), II (48·2%), and MI (40·0%), with higher cooperation rates seen in the arm using a female voice (RR: 1·18, 95% CI: 1·10–1·27, p<0.001) as compared to a male voice, and lower cooperation rates in the arm using a motivational introduction (RR: 0·83, 95% CI: 0·77–0·89, p<0.001) as compared to an informational one. In Uganda, the cooperation rates were: MV (58·2%), FV (58·8%), II (58·2%), and MI (58·8%); there was no difference in cooperation rates in the arm using a female voice (RR: 1·01, 95% CI: 0·95–1·07, p = 0.76) as compared to a male voice, or in the arm using a motivational introduction (RR: 1·01, 95% CI: 0·95–1·07, p = 0.76) as compared to an informational one.
In Bangladesh, the refusal rates were: MV (19·4%), FV (19·4%), II (16·3%), and MI (22·5%), with no difference in refusal rates in the arm using a female voice (RR: 1·00, 95% CI: 0·91–1·09, p = 0.95) as compared to a male voice, and a higher refusal rate in the arm using a motivational introduction (RR: 1·39, 95% CI: 1·27–1·52, p<0.001) as compared to an informational one. In Uganda, the refusal rates were: MV (16·3%), FV (16·6%), II (16·4%), and MI (16·5%), with no difference in refusal rates in the arm using a female voice (RR: 1·02, 95% CI: 0·89–1·16, p = 0.66) as compared to a male voice, or in the arm using a motivational introduction (RR: 1·01, 95% CI: 0·88–1·15, p = 0.94) as compared to an informational one.
In Bangladesh, the contact rates were: MV (46·5%), FV (52·1%), II (45·6%), and MI (52·3%); the contact rate was higher with the use of a female voice compared to a male voice (RR: 1.12, 95% CI: 1.07–1.17, p<0.001), and for the use of a motivational introduction (RR: 1.15, 95% CI: 1.10–1.20, p<0.001) compared to the use of an informational introduction. In Uganda, the contact rates were 63·4%, 64·5%, 61·5%, and 66·5% for MV, FV, II, and MI, respectively; it was not significantly different with the use of a female voice compared to a male voice (RR: 1.02, 95% CI: 0.98–1.06, p = 0.41) and was increased for the motivational introduction (RR: 1.08, 95% CI: 1.03–1.13, p<0.001) as compared to the informational introduction.
The refusal rates in the pooled sample in Bangladesh were: MV (19·4%), FV (19·4%), II (16·3%), and MI (22·5%); there was no difference in refusal rate when comparing female voice to male voice (RR: 1·00, 95% CI: 0·91–1·09), and an increase in refusal rate with motivational introduction as compared to informational introduction (RR: 1·39, 95% CI: 1·27–1·52, p<0.001). In Uganda, the refusal rates were 16·3%, 16·6%, 16·4%, and 16·5% respectively; there was no difference in refusal rates with use of a female voice compared to a male voice (RR: 1·02, 95%CI: 0·89–1·16, p = 0.78) or with the use of a motivational introduction as compared to an informational one (RR:1·01, 95% CI: 0·88–1·15, p = 0.94).
Sensitivity analyses
Sensitivity analyses were conducted to check for differences in respondent characteristics among complete and partial interviews as defined in the study, with no differences detected (S2 Table). Analyses were also conducted to look for differences in cooperation rate with the introduction of an interaction term: survey voice (S3 Table) and survey introduction (S4 Table), with none detected across demographic characteristics.
Discussion
We conducted this IVR survey in Bangladesh and Uganda using RDD with participants randomized to one of four combinations of male or female voice and an informational or motivational introduction to test the effects of such combinations on survey performance metrices. Demographic characteristics were comparable across the study arms in both countries. The findings in Bangladesh suggest that the use of a female voice in mobile surveys may increase contact, cooperation, and cooperation. The findings of reduced cooperation rate and increased refusal rate when using a motivational introduction in Bangladesh are in contrast to the higher response rate in Uganda with a motivational introduction. This study adds significant knowledge to the growing body of literature investigating the impact of types of voice and introductory message on the survey performance metrices.
The positive impact of female voice on overall survey performance metrices both before and after pooling by introductions in Bangladesh indicates that using a female voice could reduce the cost of conducting IVR surveys in that country. However, there was a lack of such association in participation in Uganda. Prior research in IVR optimization strove to eliminate speaker-listener gender differentials as a way of minimizing potential adverse reactions from spouses or household members, such as a husband being suspicious if a wife takes a call from an unfamiliar male voice. Sensitivity to such local gender dynamics may improve participation [39].
In Uganda, while there were higher contact, response, and cooperation rates but lower refusal rates compared to Bangladesh. It is unclear if the shift seen in Bangladesh can be attributed to survey fatigue, distrust of motivational messaging, or other survey-related factors. This difference may also be attributable to mobile network issues or survey length, both of which deserve further exploration. We should note that the contact rate findings should be interpreted with caution as they may not represent a response to the study exposure (i.e., the choice of voice gender or type of introductory message)–contact rates are an indication of the number of people that picked up the phone to answer the RDD. It is also not surprising that the differences in survey rates by gender and nature of introduction appear to be highly country-specific, given stark cultural differences between Bangladesh and Uganda [40,41]. While gender roles remain fairly traditional in both countries, there are differences within each setting that may contribute to the findings. For instance, Uganda has a roughly 68% public labor force participation rate among women, while this rate is 36% in Bangladesh [42]. With fewer women participating in the public workforce in Bangladesh, mixed gender interactions may be viewed as less appropriate. Young females (aged 15–24) in both countries, however, are 80–85% literate, revealing a much smaller difference among younger women. Despite reportedly high rates of overall mobile ownership in both countries (>65%), the 2018 GSMA Mobile Gender Gap report found a 33% gender gap in phone ownership in Bangladesh, while 2015 statistics suggest this gap is ~10% smaller in Uganda–with 77% of men owning phones vs. only 54% of women [42,43].
Given the increasing numbers of unsolicited commercial phone calls and text messages sent to mobile users in all countries [10,14], strategies to improve engagement with calls made for a collection of publicly useful data (a public good) are necessary–and may need to include ways to differentiate these calls, in addition to methods to increase already low rates of successful engagement, however, measured. Previous research has shown that survey respondents respond well to “linguistic congruence”, where the speaker in the recorded messages is from the same cultural or ethnic group [44]. In these studies, this was always the case, but it is important to note that in addition to cultural congruence, an understanding of how female or male speakers are seen as ‘trustworthy’ in a particular culture may be additionally important.
Ultimately, however, policy actions taken to reduce the background ‘noise’ represented by receiving many unsolicited advertising calls, texts, and IVR messages are likely to improve respondents willingness to engage with mobile surveys [5,14,45]. Leeper recently posited that phone survey non-response represents a “common pool resource” problem–which, similar to unregulated and unrestricted fishing can rapidly deplete the proverbial “ocean” of potential respondents [46]. It may not be plausible to consider regulating or limiting the number of phone surveys launched across the public and private sector, so efforts to motivate putative respondents to participate will need to continue.
Future qualitative research exploring the drivers of varied responses to different gendered voices and kinds of introductory messages might help with understanding the drivers of survey responses in different contexts. Other nuances that “hook” the call recipient, such as playfulness or the use of respected leaders or celebrities, have been tested in other advertising and media channels and might lend their success to this survey space as well [25,47].
The limitations of the present study also warrant discussion. In Uganda, for logistical reasons, four of the local languages and English were used to deliver the survey. Due to limited information about respondents’ survey-taking behavior, the reasons why a particular approach did or did not work in each context are difficult to glean. This may also lead to challenges with external validity, as these approaches of survey voice and survey introduction may have different effects in different contexts. There may also be confounding introduced by pooling across survey voice and survey introduction when examining survey outcomes.
Despite these limitations, this study has several notable strengths. It seeks to answer important questions about motivating survey respondents to engage in mobile-phone or phone surveys, writ large. This study uses a randomized, controlled design, and leverages a standardized questionnaire adapted from the WHO STEPwise survey questions [48], dialing participant numbers through random digit dialing. The study is adequately powered to draw inferences about the impact of these strategies (survey voice gender and survey introduction type) on survey success. These sorts of explorations into survey participation, particularly gendered effects, will help to reduce the gendered data gap in health and other data.
Conclusion
This study showed relatively better performance of using a female voice or a motivational introduction in Bangladesh. Different results were observed in Uganda, demonstrating the need for a better understanding of the impact of gendered voice and introductory message by cultural context on survey performance metrices. Further exploration of such nuances is crucial for not only improving survey participation in LMICs but also for reducing gendered data gaps in these survey modalities.
Supporting information
S1 Checklist. CONSORT 2010 checklist of information to include when reporting a randomised trial*.
https://doi.org/10.1371/journal.pone.0285155.s001
(DOC)
S1 Table. Script of motivational and information intro.
https://doi.org/10.1371/journal.pone.0285155.s002
(DOCX)
S2 Table. Demographics of complete interviews and partial interviews in Bangladesh and Uganda, n (%).
https://doi.org/10.1371/journal.pone.0285155.s003
(DOCX)
S3 Table. Sub-group analyses of cooperation rates for survey voice in Bangladesh and Uganda.
https://doi.org/10.1371/journal.pone.0285155.s004
(DOCX)
S4 Table. Sub-group analyses of cooperation rates for survey introduction in Bangladesh and Uganda.
https://doi.org/10.1371/journal.pone.0285155.s005
(DOCX)
Acknowledgments
We are grateful to those in Bangladesh and Uganda who took the time to participate in the mobile phone survey.
References
- 1. Boland M, Sweeney M, Scallan E, Harrington M, Staines A. Emerging advantages and drawbacks of telephone surveying in public health research in Ireland and the U.K. BMC Public Health. 2006;6:208. pmid:16911771
- 2. Ali J, Labrique AB, Gionfriddo K, Pariyo G, Gibson DG, Pratt B, et al. Ethics Considerations in Global Mobile Phone-Based Surveys of Noncommunicable Diseases: A Conceptual Exploration. J Med Internet Res. 2017;19:e110.
- 3. Brown W, Giguere R, Sheinfil A, Ibitoye M, Balan I, Ho T, et al. Challenges and solutions implementing an SMS text message-based survey CASI and adherence reminders in an international biomedical HIV PrEP study (MTN 017). J Biomed Inform. 2018;80:78–86. pmid:29501908
- 4. Layne J. Incredible Pros and Cons of Conducting Surveys. 2020.
- 5. Smith MG, Witte M, Rocha S, Basner M. Effectiveness of incentives and follow-up on increasing survey response rates and participation in field studies. BMC Med Res Methodol. 2019;19:230. pmid:31805869
- 6. Harrison S, Henderson J, Alderdice F, Quigley MA. Methods to increase response rates to a population-based maternity survey: a comparison of two pilot studies. BMC Med Res Methodol. 2019;19:65. pmid:30894130
- 7. Gibson DG, Pereira A, Farrenkopf BA, Labrique AB, Pariyo GW, Hyder AA. Mobile Phone Surveys for Collecting Population-Level Estimates in Low- and Middle-Income Countries: A Literature Review. J Med Internet Res. 2017;19:e139. pmid:28476725
- 8. Byonanebye DM, Nabaggala MS, Naggirinya AB, Lamorde M, Oseku E, King R, et al. An Interactive Voice Response Software to Improve the Quality of Life of People Living With HIV in Uganda: Randomized Controlled Trial. JMIR MHealth UHealth. 2021;9:e22229. pmid:33570497
- 9. Centers for Disease Control and Prevention. Youth Risk Behavior Surveillance—United States, 2017.
- 10. AAPOR Ad Hoc Committee. Spam Flagging and Call Blocking and Its Impact on Survey Research. 2022.
- 11. Pew Research Center. What Low Response Rates Mean for Telephone Surveys. 2017.
- 12. Tran MC, Labrique AB, Mehra S, Ali H, Shaikh S, Mitra M, et al. Analyzing the mobile “digital divide”: changing determinants of household phone ownership over time in rural bangladesh. JMIR MHealth UHealth. 2015;3:e24. pmid:25720457
- 13. Bastawrous A, Armstrong MJ. Mobile health use in low- and high-income countries: an overview of the peer-reviewed literature. J R Soc Med. 2013;106:130–42. pmid:23564897
- 14. Kok KF. Truecaller Insights: Top 20 Countries Affected by Spam Calls & SMS in 2019. Truecaller Blog. 2019. http://truecaller.blog/2019/12/03/truecaller-insights-top-20-countries-affected-by-spam-calls-sms-in-2019/. Accessed 2 Feb 2022.
- 15. Fleming ML. Instructional message design: Principles from the behavioral and cognitive sciences. Educational technology; 1993.
- 16. Pit SW, Vo T, Pyakurel S. The effectiveness of recruitment strategies on general practitioner’s survey response rates—a systematic review. BMC Med Res Methodol. 2014;14:76. pmid:24906492
- 17. David MC, Ware RS. Meta-analysis of randomized controlled trials supports the use of incentives for inducing response to electronic health surveys. J Clin Epidemiol. 2014;67:1210–21.
- 18. Gibson DG, Wosu AC, Pariyo GW, Ahmed S, Ali J, Labrique AB, et al. Effect of airtime incentives on response and cooperation rates in non-communicable disease interactive voice response surveys: randomised controlled trials in Bangladesh and Uganda. BMJ Glob Health. 2019;4:e001604. pmid:31565406
- 19. L’Engle K, Sefa E, Adimazoya EA, Yartey E, Lenzi R, Tarpo C, et al. Survey research with a random digit dial national mobile phone sample in Ghana: Methods and sample quality. PLOS ONE. 2018;13:e0190902. pmid:29351349
- 20. Pop-Eleches C, Thirumurthy H, Habyarimana JP, Zivin JG, Goldstein MP, de Walque D, et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS Lond Engl. 2011;25:825–34.
- 21. Edwards PJ, Roberts IG, Clarke MJ, DiGuiseppi C, Wentz R, Kwan I, et al. Methods to increase response rates to postal and electronic questionnaires (Review). Cochrane Database Syst Rev. 2009;3.
- 22. Mercer A, Caporaso A, Cantor D, Townsend R. How Much Gets You How Much? Monetary Incentives and Response Rates in Household Surveys. Public Opin Q. 2015;79:105–29.
- 23. de Tolly K, Skinner D, Nembaware V, Benjamin P. Investigation into the Use of Short Message Services to Expand Uptake of Human Immunodeficiency Virus Testing, and Whether Content and Dosage Have Impact. Telemed E-Health. 2012;18:18–23. pmid:22150712
- 24.
Thorson E, Friestad M. The effects of emotion on episodic memory for television commercials. Washington DC: Lexington Books; 1989.
- 25. Times Hindustan. PM Modi seeks support of celebs across fields to promote Swachh Bharat Abhiyan. Hindustan Times. 2017. https://www.hindustantimes.com/india-news/pm-modi-seeks-support-of-celebs-across-fields-to-promote-swachh-bharat-abhiyan/story-vQgEbxRmWgwJ3oqdHrh6rN.html. Accessed 2 Feb 2022.
- 26. Dykema J, Diloreto K, Price JL, White E, Schaeffer NC. ACASI Gender-of-Interviewer Voice Effects on Reports to Questions about Sensitive Behaviors Among Young Adults. Public Opin Q. 2012;76:311–25.
- 27. Flores-Macias F, Lawson C. Effects of Interviewer Gender on Survey Responses: Findings from a Household Survey in Mexico. Int J Public Opin Res. 2008;20:100–10.
- 28. Kane EW, Macaulay LJ. Interviewer Gender and Gender Attitudes. Public Opin Q. 1993;57:1–28.
- 29. Gibson DG, Pariyo GW, Wosu AC, Greenleaf AR, Ali J, Ahmed S, et al. Evaluation of Mechanisms to Improve Performance of Mobile Phone Surveys in Low- and Middle-Income Countries: Research Protocol. JMIR Res Protoc. 2017;6:e81. pmid:28476729
- 30. Johns Hopkins Bloomberg School of Public Health. A Randomized Controlled Trial of Narrative Voice and Introduction Content to Improve Interactive Voice Response (IVR) Survey Performance in Bangladesh and Uganda. Clinical trial registration. clinicaltrials.gov; 2018.
- 31. World Health Organization. STEPwise Approach to Surveillance (STEPS). 2022.
- 32. World Health Organization. Global Adult Tobacco Survey. http://www.who.int/tobacco/surveillance/survey/gats/en/. Accessed 1 Oct 2018.
- 33. Gibson DG, Farrenkopf BA, Pereira A, Labrique AB, Pariyo GW. The Development of an Interactive Voice Response Survey for Noncommunicable Disease Risk Factor Estimation: Technical Assessment and Cognitive Testing. J Med Internet Res. 2017;19:e112. pmid:28476724
- 34. Rutebemberwa E, Namutundu J, Gibson DG, Labrique AB, Ali J, Pariyo GW, et al. Perceptions on using interactive voice response surveys for non-communicable disease risk factors in Uganda: a qualitative exploration. mHealth. 2019;5:32. pmid:31620459
- 35. The American Association for Public Opinion Research. Standard Definitions: Final dispositions of case codes and outcome rates for surveys. 9th edition. 2016.
- 36. Rothman KJ. No adjustments are needed for multiple comparisons. Epidemiol Camb Mass. 1990;1:43–6. pmid:2081237
- 37.
Stata Corporation, College Station, Texas USA. StataCorp. 2017. 2017. https://www.stata.com/support/faqs/resources/citing-software-documentation-faqs/. Accessed 8 May 2017.
- 38. Smith T. A Revised Review of Methods to Estimate the Status of Cases with Unknown Eligibility. 2009.
- 39. Swendeman D, Jana S, Ray P, Mindry D, Das M, Bhakta B. Development and Pilot Testing of Daily Interactive Voice Response (IVR) Calls to Support Antiretroviral Adherence in India: A Mixed-Methods Pilot Study. AIDS Behav. 2015;19:142–55. pmid:25638037
- 40. Uganda Bureau of Statistics, ICF International. Uganda Demographic and Health Survey 2016.
- 41. National Institute of Population Research and Training (NIPORT), Mitra and Associates, ICF International. Bangladesh Demographic and Health Survey 2017–18. Dhaka, Bangladesh; 2020.
- 42. Okeleke K. Uganda: Driving inclusive socio-economic progress through mobile-enabled digital transformation. Mobile for Development. 2019. https://www.gsma.com/mobilefordevelopment/resources/uganda-driving-inclusive-socio-economic-progress-through-mobile-enabled-digital-transformation/. Accessed 1 Feb 2022.
- 43. Okeleke K. Achieving mobile-enabled digital inclusion in Bangladesh. 2021.
- 44. Albright K. Toward a Trustworthy Voice: Increasing the Effectiveness of Automated Outreach Calls to Promote Colorectal Cancer Screening among African Americans. Perm J. 2014;18.
- 45. Vicente P, Reis E, Santos M. Using Mobile Phones for Survey Research: A Comparison with Fixed Phones. Int J Mark Res. 2009;51:1–16.
- 46. Leeper TJ. Where Have the Respondents Gone? Perhaps We Ate Them All. Public Opin Q. 2019;83:280–8.
- 47. Brinkel J, Dako-Gyeke P, Krämer A, May J, Fobil JN. An investigation of users’ attitudes, requirements and willingness to use mobile phone-based interactive voice response systems for seeking healthcare in Ghana: a qualitative study. Public Health. 2017;144:125–33. pmid:28274374
- 48. World Health Organization. STEPwise approach to surveillance (STEPS).