Introduction

Many countries have enacted national policies to adopt universal health coverage (UHC), aligning with the World Health Organization (WHO)’s Sustainable Development Goal 3.8 to achieve UHC in all countries [1–4]. UHC strives to ensure that individuals and communities receive the complete range of essential services from health promotion through prevention, treatment, rehabilitation, and palliative care without financial hardship [1–3]. UHC incorporates two major dimensions: service coverage to promote equitable access to essential services for population health, and protective mechanisms so no person is driven into poverty as a result of seeking healthcare, especially vulnerable populations [1]. These vulnerable populations include pregnant women, adolescents, and children, who often lack the independent means to pay for required healthcare services [5–7]. The two dimensions of UHC are critical to ending preventable maternal mortality for women of reproductive age [8]. Given the priority focus for UHC on free access to maternal health-related services, WHO has developed standardized measurement indicators to monitor countries’ progress toward universally free public maternal healthcare [10, 13].

One of these global policy indicators is, “Are the following health services provided free of charge at point-of-use in the public sector for women of reproductive age?” (Table 1). This indicator measures whether 13 maternal health services are provided free to the public for women of reproductive age. Based on its relevance and importance to ensuring UHC, this indicator is a core policy indicator in the Strategies Toward Ending Preventable Maternal Mortality (EPMM) monitoring framework [9, 10]. Additionally, the indicator is one of ten measures validated as part of the multi-country Improving Maternal Health Measurement Capacity and Use (IMHM) Project [11]. Currently, measurement of the indicator uses data from laws and polices collated on the WHO’s Maternal, Newborn, Child and Adolescent Health (MNCAH) Policy Survey from the most current update in 2018 [12, 13]. However, it remains unclear whether this indicator accurately captures the country-level laws and policies or provides a valid reflection of their actual implementation during the provision of maternal health-related care.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Indicator definition and metadata.

https://doi.org/10.1371/journal.pone.0299249.t001

Establishing valid indicators is crucial to inform effectiveness of policy. How well the indicator reflects actual policies, i.e., comparison to a gold standard or criterion validity, is important because having free care policies may not be sufficient to ensure progress toward UHC for maternal health [14]. Indeed, such policies may not translate to the intended care at point-of-use [15]. Women may accrue formal and informal charges for services that are reportedly free at the policy level [7, 15, 16]. Charges may be consequent to deficiencies in the health system, including shortage of essential supplies that may force providers to charge for services designated as free [17, 18], or reflect a demand for informal payments [17, 19]. Thus, the indicator may have weakened construct validity, or ability to capture the concept of free care for maternal health services [14]. In addition, there may be a tendency for the poor, illiterate, and less advantaged to be unfairly charged, moving away from rather than toward equity in UHC for maternal health [20–22].

These limitations with policy indicators underscore a need to empirically determine whether this indicator provides a valid measure of both policy and practice related to maternal health UHC, specifically regarding access to essential services and freedom from financial harm. Further, equity is a key aspect of UHC, so it is important to assess whether the indicator is a valid measure of clinical and financial coverage of essential care for all women who need maternal health services. Thus, this multi-country study compared the value of the indicator as reported globally to evidence collected in each country and identified variance that could threaten the measure’s validity. Our findings provide important evidence to support strengthening this indicator.

Methods

Results

Survey demographics

Table 2 displays the characteristics of survey respondents in each country.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Characteristics of survey respondents.

https://doi.org/10.1371/journal.pone.0299249.t002

Argentina.

We interviewed a total of 1046 women and accompanying persons in Argentina. The sample was almost evenly distributed across the four provinces, and 45.4% were aged 25–34 years. Almost all of the women could read and write (99.2%), 60% had completed secondary education, 73.3% did not have health insurance, and 30.6% were in the middle wealth quintile. The most frequent services obtained were family planning services (11.9%) and antenatal care (10.1%). We interviewed 22 financial officers from urban public facilities in Argentina, 13 of whom (59.1%) were stationed at primary health facilities, with a mean ± SD working experience of 23 ± 9.8 years.

Ghana.

In Ghana, we interviewed 923 women almost evenly distributed across the districts. Almost half (46.2%) were 25–34 years old, 59.8% could not read and write, 36.8% had no formal education, and 93.4% had health insurance. The sample was evenly distributed among wealth quintiles, with most participants (28.4%) in the poorest quintile. The most frequent services obtained were antenatal care (37.6%) and pharmaceutical and other medical supplies (24.8%). We interviewed 115 financial officers in Ghana representing 145 health facilities, 103 of these facilities (71.0%) were in rural settings and 144 (99.3%) were primary care facilities. Financial officers had a mean ± SD working experience of 9.0 ± 7.5 years.

India.

In India, we interviewed 1102 women and accompanying persons. The sample was evenly split across the four districts, with 53.0% aged 25–34 years. Most participants (88.9%) could read and write, 46.9% had a secondary-level education, and 84.5% had health insurance. Most participants (55.6%) were in the richest quintile. Services were well-distributed in India, with the most participants receiving antenatal care (12.3%) and family planning (8.6%). We interviewed a total of 158 financial officers, 77.2% of whom worked in rural facilities and 53.2% in primary care facilities, with mean ± SD working experience of 11.2 ± 8 years.

Concurrence between country-level policy and MNCAH policy survey

Table 3 compares source data from all three countries with reported global data. In Argentina, country-level policies designated 12 out of the 12 applicable services as free. This matched what was reported in the WHO MNCAH Policy Survey. In Ghana, all 13 services were designated as free by country-level policies and the WHO MNCAH Policy Survey. In India, country-level policies designated 11 out of the 13 services as free for all women; insecticide-treated bed nets and testing and treatment for syphilis were free for selected populations (those living in areas where vector-borne illnesses and syphilis were endemic). However, the WHO MNCAH Policy Survey designated all 13 services as free.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Validation of results from policy review with interviews of women and financial officers.

https://doi.org/10.1371/journal.pone.0299249.t003

Payments associated with maternal health services designated as free by policy

Table 3 also displays data on payments made for services desginated as free in all three countries and Table 4 displays the reasons provided for payments. In Argentina, financial officers reported high awareness of free care laws for the 12 services, ranging 59.1%–100%. Officers also reported a low prevalence of women seeking care in their facilities ever needing to make out-of-pocket payments associated with the 12 free services, ranging from 4.6% for maternal immunization services, childbirth (vaginal and cesarean birth), management of birth complications, pharmaceutical products/other supplies for childbirth, postnatal care, testing and treatment of STIs/syphilis/HIV, and cervical cancer screening to 9.1% for family planning and antenatal care. The reasons they reported for charges were mainly for purchase of supplies (13.7%), registration fees (13.7%), and laboratory fees (13.7%).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Reasons for payments associated with services designated as free, countries.

https://doi.org/10.1371/journal.pone.0299249.t004

Women in Argentina reported having costs associated with all 12 services designated as free, with prevalence of associated out of pocket costs ranging from 8.6% for management of other birth complications to 26.0% for cesarean sections. Of the 238 times women in Argentina reported costs associated with services designated as free, 25.6% were payments for purchase of supplies and 57.1% for travel costs.

In Ghana, financial officers reported awareness of free care laws ranging from 6.9% for cervical cancer screening to 94.5% for insecticide-treated bed nets. Officers reported ever needing to charge for 12 out of 13 services (except insecticide-treated bed nets), with a prevalence ranging from 0.69% for postnatal care for mother and immunization services during pregnancy to 64.1% for family planning. Financial officers reported fees were mainly charged for purchase of supplies (48.1%) and services rendered (27.3%).

Women in Ghana reported being charged for 11 out of the 13 services designated as free (cervical cancer screening was not routinely offered in districts, and no woman was charged for cesarean section), with prevalence of fees ranging from 0.5% for immunization services to 78.4% for family planning. Of the 490 times women in Ghana reported being charged for services designated as free, 39.6% were payment for purchase of supplies and 33.5% were payment for services rendered.

In India, financial officers reported awareness of free care laws ranging from 88.6% for cesarean delivery to 100% for testing and treatment of HIV. Officers reported ever charging for all 13 services, with a prevalence ranging from 6.3% for insecticide-treated bed nets to 29.7% for testing and treatment of HIV. Financial officers reported fees charged were mainly for travel costs (40.2%), food costs (26.2%), and registration fees (21.2%).

Women in India frequently reported been charged for all 13 services designated as free, with prevalence of fees ranging 60.7% for insecticide-treated bed nets to 94.4% for postnatal maternal care. Of the 2,539 times women in India reported being charged for services designated as free, 32.8% were payment for travel costs and 20.8% for food costs.

Payments associated with services designated as free based on women’s demographic characteristics

Table 5 displays the association between payment for services designated as free and women’s characteristics. Women’s out of pocket expenditures when accessing services designated as free varied significantly by various standard EPMM equity stratifiers across the three countries.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Payment for services designated as free based on women’s social and EPMM equity stratifiers.

https://doi.org/10.1371/journal.pone.0299249.t005

In Argentina, women’s expenditures varied significantly by province (p < 0.001), educational status (p = 0.007), and wealth quintile (p = 0.045). In Ghana, women’s expenditures varied by district (p < 0.001) and educational status (p = 0.011). In India, women’s expenditures varied by residence district (p < 0.001), educational status (p < 0.001), residence (p = 0.022) and age (p = 0.019).

Discussion

We assessed validity of a global policy indicator on free care for maternal health services, a measure of UHC intended to monitor access to essential public health service without financial burden for all women of reproductive age. In our desk review, in Argentina and Ghana, we found consistency between existing country-level policies and those reported globally in the WHO MNCAH Policy Survey; in India, we found minor discrepancies between country-level policy on record and what was reported in the WHO MNCAH Policy Survey. However, interviews with women and financial officers revealed that women often had out of pocket expenditures associated with accessing services designated as free by policy. Expenditures were reported for direct and indirect costs of care, and included both formal payments reflecting health system deficiencies, and informal payments not acknowledged by the facility management. Additionally, reported payments varied differentially by some of the demographic characteristics reflected in the standard EPMM equity statifiers across the three settings. Our findings echo existing concerns from stakeholders on the construct validity of this indicator in accurately assessing UHC for maternal health [19, 24, 25].

The consistency of global database information for Argentina and Ghana affirms the criterion validity of this indicator in these settings. Countries are required to submit documentation on free care policies to support their responses [13], indicating that the validation process was thorough for Argentina and Ghana. However, for India there was mismatch between information in the global database and country-level findings. As India also submitted documentation of its reference policies, this mismatch may indicate a simple error. WHO has previously reported issues in its validation of countries’ reported policies [13]. The inconsistency could also be due to discrepancy in policy interpretation. It is worthwhile to note that the two services with mismatch were both designated as free only for select populations at the country level. In our study, we interpreted this to mean the services were not free for everyone in the country; however, it is possible that in the global database this was interpreted as free for all who need it (i.e., those living in vector-borne and syphilis endemic areas). Regardless of the reason, the discrepancy in reported policies between the WHO MNCAH Policy Survey and national-level findings for India raises concerns about the indicator’s criterion validity in this setting. Moreover, numerous studies demonstrate that policy-level declarations do not always reflect the reality at the service provision level [7, 16, 26].

Furthermore, our study uncovered gaps between policy and practice. Overall, we found that women had out of pocket expenditures when accessing services designated as free, as established from triangulation of different sources of data. Women who accessed services reported having associated out of pocket expenditures for all 12 component services in Argentina, all 13 component services in India, and for 11 of 13 component services in Ghana. The expenditures reflect both direct and indirect costs associated with the services obtained, including purchase of supplies across all settings, as well as travel and food costs in Argentina and India. Further, our findings suggest charges that were both formal and informal, for example, service and registration fees in Ghana and India and payments to health workers in India. This finding is consistent with previous research in developing countries showing that women are charged both formally and informally for services designated as free [7, 15, 16]. Although financial officers reported their awareness that the services were free by policy, they also reported that necessity may dictate the existence of women’s out of pocket expenditures. A major reason for this need to charge was for purchase of supplies, in line with previous findings indicating that shortages of essential supplies may force providers to charge for services that should be free [17, 18]. These findings compromise the validity of the indicator as a measure of free universal access to essential maternal health services by reproductive-age women without financial burden in all our study settings.

Generally, few financial officers reported there ever being a need for women to make out of pocket payments when accessing free services. This may be because financial officers feel they are violating policies by charging and thus are less likely to freely report the charges. However, where financial officers did report ever needing to charge for free services, their responses were generally aligned with women’s reports, except in the case of direct payments to healthworkers or fees for services, which likely reflect informal payments. This reiterates the need to ensure an enabling environment for implementation of free care laws [27–29].

We evaluated the validity of the measure by collecting data directly from end-users to corroborate information reported at the policy level—our results suggest that both health system administrators’ and women’s experiences are critical to ensuring the construct validity of the policy indicator, that it captures the intended concept of universal health coverage for maternal health fully, accurately, and reliably from both the supply and demand side. Indeed, other studies have called for changes in data sources and recommended measurement indicators evolve over time to reflect changing policy priorities [30, 31]. Using both end-user data sources also captured the range of formal and informal charges for services, which is vital to help target interventions and policy refinements where necessary.

Another way we tested the indicator’s validity was to change the estimation method. We assessed specific service categories, while original computation of the indicator used categories of women who were charged for free services to assess whether care was free for all, some, or no types of women. Our approach assessed free care by service category and further analyzed findings to determine equity. We chose assessment at the service level because some services are more likely to be charged in certain settings and may have greater costs compared to others. As demonstrated from our findings, the specific services women were charged for differed across the three settings. Evidence shows that targeting interventions to specific populations is less effective than universal coverage of services with human rights implications [32, 33]—it is more beneficial for a particular number of services to be free for all rather than for all services to be free for targeted groups only.

Collecting data at the service level enables disaggregation by individual-level equity factors. Our results showed that services were differentially charged to women based on wealth quintile, district/province, education status, residence, and age within the three settings. Differential charges indicate inequity, with major implications for UHC [16, 26]. Although not directly assessing charges or user fees, previous studies have established inequities in access and utilization of maternal and child health services, with some related to rural–urban differences in costs associated with accessing services in Ghana, Malawi, India, and South Africa [20, 22, 34]. Other studies in Ghana and Burkina Faso have established equity concerns for accessing maternal health services [19, 35, 36]. As equity is an important aspect of UHC, these findings challenge the validity of the indicator as a measure of this construct.

Another concern that challenges the validity of the indicator is the inability of the current measurement to capture all dimensions of UHC. In our study, we captured all costs associated with accessing maternal healthcare by women (formal, informal, direct and indirect costs). Capturing other costs is directly in line with the second dimension of UHC which looks at protective mechanisms so no person is driven into poverty as a result of seeking healthcare [1]. The current measurement of this indicator from the policy level is unable to capture this dimension of UHC because existence and even full implementation of laws on free care only ensure women will not be charged at point of use. Meanwhile, women will continue to suffer financial harm in accessing services due to need to cover associated costs including transportation and food costs as demonstrated by our findings. This further challenges the construct validity of this indicator in accurately measuring UHC for maternal health.

A major strength of this study is our rigorous approach to validate the indicator. Triangulating findings from country-level policies with interviews of women and financial officers and collecting data directly from end-users provided a fuller account of the reality of payments associated with free care, identifying threats to validity in the current measurement of the indicator. Our findings suggest that this policy indicator may need to be complemented by measures that monitor implementation of the policy at the point-of-use by service users (demand side) and at the service delivery level (supply side) for optimal construct validity.

This study also has important limitations. Our findings from self-reported key variables could possibly be affected by courtesy bias, especially in the case of interviews with women. Given that interviewed women were likely to continue receiving services at the same facilities, they may have under-reported charges [37]. Nonetheless, our findings showed significant charges levied to women, even if underreported. In Argentina, due to the inability of women to discern all factors contributing to the categorization of wealth, there were some missing values for the wealth index. However, since this affected all provinces, the effect is unlikely to significantly change our findings in this setting. Further, the data collection method in each country varied for women and financial officers (in-person, telephone, or online interviews), which could have resulted in some information bias. Due to the COVID-19 pandemic, routine service provision was altered and for some services we could not interview women because the services were not offered during the pandemic (e.g., screening for cervical cancer in Ghana) which could have introduced some information bias. Additionally, some districts did not offer some services, limiting the service sample in those districts. Our findings on equity could also be possibly affected by systematic differences in self reporting of some costs due to price sensitivity (eg; transportation costs). For example, it is likely that all (or nearly all) women would have had to pay for transport to the hospital as few live within walking distance. However, only a portion of the sample reported transit costs. It may be that the poorest women are most likely to register this as a cost due to higher price sensitivity. As we are unable to differentiate between these from our data, this limitation should be considered in conducting needed further studies to operationalize data collection for routine measurement of an indicator that tracks implementation of free care at the service level.

Conclusion

Our results showed that the global WHO MNCAH Policy Survey largely accurately reported free care laws and policies. Nevertheless, we found that women had out of pocket expenditures that reflect both formal and informal charges and bore both direct and indirect costs associated with services designated as free by country-specific policies. Therefore, our results indicate the policy indicator of free care for maternal health services did not provide a valid reflection of UHC in India, Ghana, and Argentina.

Acknowledgments

The authors would like to thank the following people, without whose efforts the publication of this manuscript would not have been possible:

In Argentina, we gratefully acknowledge the support of the National Directorate of Maternal, Child and Adolescent Health and the Directorate of Sexual and Reproductive Health of the Ministry of Health of the Nation. We commend the compromise and dedication of the provincial teams, members of the Maternal and Child Health Programs of the Provincial Ministries of Health: Dr. Adriana Martirena, Dr. Daniel Nowacky, Dr. Adriana Allones, Marta Ferrary, Dr. Claudia Castro, Ana Seimande, Antonio Tabarcachi, Noelia Coria, Cintia Jacobi, Laura Soto, Dr. Mara Bazán, Dr. Susana Velazco, Dr. Patricia Leal, and Marcela Tapia. Finally, we would like to express our deepest gratitude to all of the health workers who participated in the study as data collectors, working through the height of the COVID-19 pandemic in Argentina.

In Ghana, we gratefully acknowledge the the support of the Director General–Ghana Health Service–Dr. Patrick Kuma-Aboagye and the Family Health Division of Ghana Health Service; Dr. Ernest Konadu Asiedu, Ms. Roberta Asiedu, Dr. Margretta Chandi and Ms. Catherine Adu Asare; Ms. Philomina Agbasi, Ms. Nyarko Agyemang-Duah, Mr. Tony Godi; and all regional and district health workers and field teams for their persistence in data collection and entry despite the challenges.

In India, we gratefully acknowledge the support of Dr. Dinesh Baswal, Ex Deputy Commissioner at Maternal Health Division, Ministry of Health & Family Welfare, India; the Mission Directors, State Health Departments of Tamil Nadu and Uttar Pradesh, and the District health Officials of study districts. We also acknowledge the support of Dr. Manju Chhugani and Dr. Renu Kharb for their guidance in review of the secondary data on many indicators. Finally, we sincerely thank the district field teams for their untiring efforts and adaptation to new methodologies to collect good quality data, in the midst of COVID in India. We also thank all the health workers and facility staff who participated in the study despite their busy schedules due to COVID situation.