The quality of skilled birth attendants in Nepal: High aspirations and ground realities

Background While Nepal’s maternal mortality ratio (MMR) has improved overall, the proportion of maternal deaths occurring in health facilities and attended to by skilled birth attendants (SBAs), has nearly doubled over 12 years. Although there are numerous socioeconomic, environmental and other factors at play, one possible explanation for this discrepancy between utilization of skilled maternal care services and birth outcomes lies in the quality of care being provided by SBAs. The objective of this study is to determine how competent SBAs are after training, across multiple settings and facility types in Nepal. Methods We used a quantitative cross-sectional analysis to evaluate a sample of 511 SBAs, all female, from 276 sub-health posts (SHP), health posts (HP), primary healthcare centers (PHC), and district and regional hospitals in the mountain, hill, and terai districts of Nepal. Any SBA actively employed by one of these health facilities was included. SBAs who had received less than three months of training were excluded. Outcomes were measured using SBAs’ scores on a standardized knowledge assessment, clinical skills assessment, and monthly delivery volume, particularly as it compared with the WHO’s recommendation for minimum monthly volume to maintain competence. Results SBAs on average exhibit a deficiency of both knowledge and clinical skills, failing to meet even the 80-percent standard that is required to pass training (knowledge: 75%, standard deviation 12%; clinical skills: 48%, standard deviation 15%). Moreover, SBAs are conducting very few deliveries, with only 7 percent (38/511) meeting the minimal volume recommended to maintain competence by the WHO, and a substantial fraction (70/511, 14%) performing an average of no monthly deliveries at all. Conclusions Taken together, our findings suggest that while countries like Nepal have made important investments in SBA programs, these healthcare workers are failing to receive either effective training or sufficient practice to stay clinically competent and knowledgeable in the field. This could in part explain why institutional deliveries have generally failed to deliver better outcomes for pregnant women and their babies.

Introduction socioeconomic, and other factors likely play a role in the discrepancy between increased facility births and maternal mortality, including that more women may go to a facility when they have complications but too late for SBA intervention to be effective [17].
The Nepal SBA training program is a 3-month in-service training program for nurses, doctors, and midwives, based on a checklist of 27 core skills and abilities for safe birth, which includes management of normal deliveries as well as potential complications. The objective of this study is to measure the level of knowledge, the degree of skill retention, and the productivity of a sample of Nepal's SBAs. We were particularly interested in post-training follow up in the field: what do SBAs actually do after SBA training? We thus posed three questions: 1) what level of knowledge do SBAs exhibit, and which demographic and environmental factors correlate with higher knowledge scores? 2) What level of clinical skills do SBAs exhibit, and which factors correlate with higher skills? And, given the proven association between case volume and patient outcomes, 3) how many deliveries are SBAs performing per month, and how many of them are meeting World Health Organization (WHO) guidelines for minimum volume to maintain competence [18][19][20][21]? What are predictors of higher delivery volume among SBAs?

Methods
We used a quantitative retrospective cross-sectional analysis to evaluate a sample of 511 SBAs in Nepal. The Nick Simons Institute (NSI), via its Follow-up Enhancement Program (FEP), and the Government of Nepal's National Health Training Center (NHTC) collected data from 2013 to 2016, attempting to follow up with 30% of SBA trainees, per NHTC recommendation. Districts were purposively sampled to represent the three ecological zones: hill, mountain, terai. We randomly chose districts from each eco-zone. The unit of randomization was the district, but individual SBA counts were used for analyses. We chose 3 districts in the Mountain region, 5 from the Hills and 7 from the Terai (S2 Table). Higher numbers of districts were chosen in the Terai versus the Mountains because a larger proportion of the Nepalese populations lives in the Terai. We then visited all of the health facilities (hospitals, PHCs, HPs, SHPs) in a given district that had skilled birth attendants working at them and carried out assessments on every one of the SBAs. We excluded SBAs who had received less than three months of training to ensure only fully trained SBAs were included, and only female SBAs are included because there were no male SBA trainees in the program. 511 SBAs remained after all inclusion and exclusion criteria were applied. We obtained ethical approval from the Nepal Health Research Council of Nepal (reference number 2410).

Ethics statement
Please note that this was a retrospective observational study of data already collected by NSI in the context of the FEP. Data were not collected for the sole purpose of research but rather as part of the FEP program carried out in conjunction with the National Health Training Center of Nepal. Approvals for the FEP program were obtained from the National Health Training Center, Department of Health Services. Since the FEP program was carried out in collaboration with the National Health Training Center, data collection was a routine part of the government program and individual consent from SBAs was verbally obtained. assistance of NSI and public health nurses. The SBA FEP tool was drafted by SBA trainers, Support for Safer Motherhood Project, and Family Health Division and was pretested in five districts, the findings of which were disseminated to stakeholders and used to revised the tool. The assessment, which included background information, knowledge and clinical assessment, clinical decision making, infection prevention, and an enabling environment checklist, took approximately 5 hours for each SBA to complete. SBA trainers also conducted guided qualitative interviews with the participants, but this information was not robust enough to analyze for the purposes of this study. SBA trainers were not blinded, as the FEP was not performed only for research purposes.
We collected descriptive information including demographic characteristics like SBAs' age, position (auxiliary nurse midwife versus staff nurse), recruitment status (permanent position versus temporary hire); level of experience (the number of months of both total nursing and SBA experience); training site type (whether continuously monitored and supported by the Nick Simons Institute, or not; support entailed financial and technical support with frequent monitoring visits to ensure a high level of quality of care and teaching at the sites); location (mountain, hill or terai district) and facility (hospital versus lower level facility like PHC, HP and SHP); and environmental characteristics: the presence or absence at the SBA's facility of each of a comprehensive set of 74 structural and inventory items that enable basic emergency obstetric care. We assessed level of knowledge of procedures and managing complications using a 20-question multiple-choice survey. We assessed level of clinical skills using standardized checklists on anatomical models and cases for use of the partograph, normal and vacuum delivery, newborn resuscitation, referral, pregnancy complication management. Normal delivery, vacuum delivery, and newborn resuscitation were assessed via demonstration on a mannequin; and partograph, postpartum hemorrhage management skills, eclampsia, and shock management were assessed via case-based discussions. We developed both the knowledge survey and clinical skills checklists using a committee of experts and WHO and JHPIEGO guidelines. The enabling environment was evaluated via FEP trainer assessment of a comprehensive checklist including infrastructure, equipment, and drugs at each facility. We assessed the total monthly deliveries for each SBA by reviewing each facility's delivery logs, and collected data on number of normal deliveries, vacuum deliveries, breech deliveries, and the management of any of the following 7 complications over a 3-month period: postpartum hemorrhage, antepartum hemorrhage, eclampsia, episiotomy, manual removal of the placenta, post-abortion care, and newborn resuscitation.

Data analysis
Data collected from the FEP tool from 2013 to 2016 was later reviewed by NSI and NHTC staff for analysis, as this data was initially collected with the intention of quality improvement and coaching or enhancement of SBAs, rather than research. While both qualitative and quantitative data were collected as part of the FEP, only quantitative data were analyzed for the purpose of this study. We recorded all quantitative data in Excel and analyzed it with Stata Version SE14. Our main outcomes were SBAs' 1) level of knowledge, 2) level of clinical skills and 3) total number of monthly deliveries. For level of knowledge, we calculated a mean percentage score, range, and standard deviation based on the number of questions answered correctly. For level of clinical skills, we calculated mean percentage scores with range and standard deviation for each of the eight domains tested, based on the number of checklist items within each domain that were successfully demonstrated. For monthly deliveries, we calculated the percentage of SBAs in each of four categories: 0, up to 4, between 4 and 15, and at least 15 monthly deliveries. We tabulated similar statistics for delivery complications. For the enabling environment, we conducted a literature review to identify which of the 74 indicators studied were deemed most essential in similar settings. We selected a final list of 17 indicators and calculated a mean percentage score according to the fraction of these present at any SBA's health facility. This list spanned three categories: 1) general requirements-electricity, water, and a toilet inside the labor room; 2) routine delivery care-a partograph, fetoscope, baby weighing machine, blood pressure instrument, soap, gloves, an autoclave, and a standard delivery set; and 3) basic emergency obstetric and newborn care-oxytocin, magnesium sulfate, a vacuum delivery set, tear repair set, newborn resuscitation set, and I.V. set.
We used the t-test to compare continuous variables, and the chi-square test to conduct between-group comparisons for categorical variables. We performed univariable and multivariable linear regressions to identify independent factors associated with 1) knowledge scores, 2) clinical skills scores, and 3) volume of monthly deliveries. To compare results between hospitals and lower-level facilities, we performed a sub-group analysis of the delivery volume outcome by facility type.

Patient involvement and dissemination
The focus of this study was the healthcare worker, specifically the skilled birth attendant. Thus, we did not involve patients in the development of the research question or in the design or writing up of the study. We will make the study's findings publicly available on the Nick Simons Institute's website. Additionally, we will disseminate the results of the study to the relevant governmental bodies in Nepal, all in coordination with the National Health Training Center.

Demographic characteristics of SBAs
511 SBAs were sampled from 276 health institutions, across 15 districts, in all three of Nepal's ecological zones (S1 Table). Of the total 511 SBAs, 75% of them worked in lower-level facilities (PHC, HP, or SHP) and 25% worked in hospitals. More than half (57%) were between the ages of 20 and 40 years, with the mean age of 33 years ranging 19-59 years ( Table 1).
91% of SBAs were auxiliary nurse midwives (ANMs), and 9% were staff nurses. Staff nurses were approximately 15 times more represented in hospitals than in lower-level facilities (29% vs 2%).
SBAs were split nearly evenly between permanent and temporary nurses overall-permanent nurses made up 51% of SBAs and temporary nurses made up 49% of SBAs. However, hospitals had a higher percentage of permanent SBAs (59%) compared to temporary ones (41%) while the opposite was true for lower level facilities (permanent 48%, temporary 52%).
On average SBAs had accumulated nearly ten years of nursing experience, three of which comprised SBA experience. At both hospitals and lower-level facilities, less than a fifth of SBAs had been trained at a continuously monitored and supported training site, though hospitalbased SBAs were 6% more likely than those at lower-level facilities. At the time of study, 56% of SBAs were working at facilities in the Terai, 31% in the Hills, and 14% in the Mountains.

Enabling environment indicators
An examination of each SBA's work environment revealed a high mean score on the index of enabling environment indicators determined to be most essential to quality maternal care. Specifically, SBAs had access to on average 83% [SD 10] (Table 2) of the 17 key enabling environment indicators, with at least 99% of them in settings with a fetoscope, baby weighing machine, gloves, and oxytocin in stock. By contrast, only 23% of SBAs could point to a toilet in the labor room, while only 27% could obtain a standard delivery set on-site. Surprisingly, only 78% of SBAs worked in a setting with readily available drinking water. Between facilities, hospitals achieved total environmental index scores on average eight percent higher (89% [SD 7]) than their lower-level counterparts (81% [SD 10]). The greatest differences emerged for availability of basic infrastructural components like electricity (91% at lower-level facilities versus 100% at hospitals, water (75% versus 88%), and a toilet in the labor room (22% vs 27%) as well as the requirement for routine and basic emergency obstetric care: partograph (83% at lower level facilities compared to 98% at hospitals), standard delivery set (24% vs 38%) and vacuum delivery set (35% vs 94%).

Knowledge of SBAs and predictors of knowledge
SBAs achieved a total mean score of 75% [SD 12] on the written knowledge assessment section of the FEP data collection tool, with roughly comparable performance between hospitals and lower-level facilities (Table 3). On multivariable analysis, SBAs' age, position and access to key enabling environment indicators proved predictive of their knowledge scores (Table 4). SBAs above the age of 50 received knowledge scores that were on average 8.88 percentage points lower than those under 30 years (95% CI -15.48 to -2.28, p = 0.009). Auxiliary nurse midwives received scores that were on average 4.72 percentage points lower than staff nurses (95% CI -8.59 to -0.86, p = 0.017). On average, a one-point increase in the enabling environment index

Clinical skills of SBAs and predictors of clinical skills
Considered both independently and relative to their knowledge assessments, SBAs performed poorly on their clinical skills, with a total mean score of 48% [SD 15] across facilities (Table 3). Referral 36 [24] 32 [23] 35 [24] Shock management 33 [20] 40 [22] 35 [21] Post-partum hemorrhage management 83 [17] 87 [15] 84 [17] Values presented as Mean [SD] https://doi.org/10.1371/journal.pone.0214577.t003 The quality of skilled birth attendants  (Table 5). SBAs above the age of 50 received clinical skills scores that were on average 11.64 percentage points lower than those under 30 years (95% CI -19.76 to -3.52, p = 0.005). Auxiliary nurse midwives posted clinical skills scores that were 8.56 percentage points lower than staff nurses (95% CI -13.32 to -3.81, p<0.0001). SBAs trained at continuously monitored and supervised training sites achieved average clinical skills scores 4.85 percentage points higher than those trained at other sites with lower levels of monitoring and supervision (95% CI 1.43 to 8.28, p = 0.006). On average, a one-point increase in the enabling environment index for an SBA's health facility correlated with a 2.31-percentage point increase in clinical skills scores for that SBA (95% CI 1.47 to 3.14, p<0.0001).

Deliveries and predictors of deliveries
Each month, 1108 deliveries occurred in hospitals, 196 in PHCs, 957 in HPs, and 25 in subhealth posts across all three ecological zones (S2 Table). An analysis of monthly delivery volume revealed that only 9% of SBAs on average were meeting the WHO's guideline of 15 deliveries per month to maintain competence. This number was significantly lower for SBAs working at lower-level facilities than for those at hospitals: 4% versus 18%, respectively (Fig 1). By contrast, a substantial fraction of SBAs-21% at lower-level facilities and 16% at hospitalsconducted zero monthly deliveries. Approximately 42% of SBAs performed between one and four monthly deliveries on average. Of the SBAs who performed an average of zero deliveries per month, mean age was slightly higher than for the population at large, standing at 37 years [9], compared with 33 years for SBAs in aggregate (Table 6). Average total nursing and total SBA experience were accordingly greater as well, at 160 months [105] and 41 months [29]. 91% of this subset of SBAs were The quality of skilled birth attendants in Nepal: High aspirations and ground realities ANMs, and 85% were trained at sites with lower levels of monitoring and supervision, similar to the aggregate sample. Slightly more than in aggregate (59% vs 53%) practiced in the Terai district. The most significant difference lay in recruitment: compared with temporary contract nurses, permanent nurses comprised a disproportionately higher percentage of SBAs conducting zero monthly deliveries than they did of SBAs in aggregate (64% vs 51%, respectively). Over a three-month period, a total of 821 complications were managed in our sample (S3 Table). The majority of SBAs managed zero complications (Fig 2), with this proportion higher for those based at lower-level facilities (76%) than at hospitals (48%). By contrast, only 1 percent of SBAs at lower-level facilities and 6 percent of those at hospitals managed at least 15 complications over the three months. The most frequently managed complications were episiotomies and newborn resuscitations; the least frequently managed: antepartum hemorrhage and post-abortion care.
On multivariable analysis, statistically significant predictors of SBA monthly delivery volume were training at a continuously monitored and supervised site and the type of facility and type of district in which SBAs practiced (Table 7). Of note, this multivariable analysis included the additional control variable of expected number of deliveries at the SBA's health facility, On subgroup analysis by facility type, there were significant differences between hospitals and lower-level facilities in those factors that proved predictive of monthly delivery volume (Table 8). For lower-level health facilities, an SBA's recruitment status, access to enabling environment indicators, and geographical district of practice were positively predictive of monthly delivery volume; age over 50 years was negatively predictive. For example, at lower level health facilities, temporary SBAs conducted 1.95 more deliveries than permanent SBAs (p = 0.003); SBAs over the age of 50 conducted 3.6 less deliveries than SBAs less than 30 years old (p = 0.026); a one point increase in the enabling environment index meant a 0.46 increase in number of deliveries (p = 0.007); and SBAs based in the Terai conducted 2.90 more deliveries than those based in the mountains (p<0.0001). For hospitals, by contrast, only an SBA's geographical district of practice and training at a continuously monitored and supervised site were positively predictive; no factors were negatively predictive. For example, at hospitals, SBAs who were trained at a continuously monitored and supervised site carried out almost 18 The quality of skilled birth attendants in Nepal: High aspirations and ground realities The quality of skilled birth attendants in Nepal: High aspirations and ground realities more deliveries than those trained at other sites with lower levels of monitoring and supervision (p<0.0001) and SBAs based in the Terai carried out almost 12 more deliveries than those based in the mountains (p = 0.052).

Discussion
Our study evaluated the competence and practice levels of a sample of skilled birth attendants in Nepal. We found that SBAs on average exhibit a deficiency of both knowledge and clinical skills, scoring poorly on assessments for both. In fact, across both knowledge and clinical skills, assessment scores failed to meet even the 80-percent standard that is required to pass training. This finding remained true no matter how recently training had concluded, suggesting not merely an attrition of skills but a failure to effectively learn them in the first place. This may indicate failure of the training program to effectively train the SBAs due to problems with the training methodology itself. We found also that SBAs are conducting very few deliveries, with only 7 percent meeting the minimal volume recommended to maintain competence by the WHO-and a substantial fraction performing an average of no monthly deliveries at all. Predictive of both their delivery volume and their knowledge and clinical skills scores was the quality of SBAs' enabling environment: higher scores on the enabling environment index correlated with both higher knowledge and higher clinical skills assessment scores, in both hospitals and lower-level facilities. Higher enabling environment scores were also correlated with greater delivery volumes for SBAs in lower-level facilities. Finally, continuously monitored and supervised sites produced SBAs with higher average delivery volumes than other sites with lower levels of monitoring and supervision. Taken together, our findings suggest that current skilled birth attendant training alone might not suffice to ensure adequately productive and competent birth attendance in the field. Continuous monitoring and supervision of SBA training sites may improve the quality of training. Further support of health facilities via structural improvements that create a more enabling work environment may in turn improve the overall quality of work of skilled birth attendants in the field.

Comparison with other studies
The first study to systematically assess SBA competencies was conducted from 2001 to 2002; a survey of 166 SBAs in four countries, it revealed similarly poor performance on clinical skills and knowledge. But it did not further analyze predictors of poor performance, the environmental factors under which SBAs work or their actual productivity in the field in terms of the number of deliveries conducted [5]. Other studies have built on this work, including one of 104 SBAs conducted just a year afterward in Nepal [6][7][8][9][10]. This study found that SBAs who underwent refresher trainings after some time in the field demonstrated greater clinical competence than those who did not, while also citing structural and environmental factors as barriers to SBA performance [6]. A more recent study in Nepal investigated this impact of environmental factors further, using semi-structured interviews and focus group discussions with SBAs to identify specific environmental constraints to maximal performance in the field [22]. None of these studies, however, explored predictive factors for the key outcomes of posttraining delivery volume, knowledge, or clinical skills among SBAs. Nor has any study investigated SBA delivery volume as a core outcome measure, or used it as a surrogate for quality; yet we know from prior studies that obstetrical case load, and provider volume more generally, can be associated with better outcomes [19][20][21]. Critically, while at least a few have observed that SBAs are performing lower delivery numbers than expected, none has highlighted that a large proportion conduct a monthly average of zero deliveries, inspecting specifically the characteristics of this population. And while some have emphasized the importance of an adequate enabling environment for SBAs, none has sought to quantify this, nor rigorously correlated this with SBA competence and productivity. Our study, thus, contributes actionable insights about the efficacy of SBA training programs and the realities of SBAs' capacity in the field.

Conclusions and policy implications
Our study offers several important policy implications. Firstly, and perhaps most importantly, regular post-training follow-up is critical to assuring continued competence of SBAs. Indeed, none of our own findings would have come to light without the post-training follow-up conducted by Nepal's National Health Training Center in coordination with NSI. In Nepal, the national policy since 2007 has been to subject at least 30 percent of SBA trainees to follow-up visits. Amidst resource constraints, however, only one such assessment prior to this study was conducted in 2009. If training programs are to be held accountable and SBAs are to be relied upon for quality maternal care, more frequent, nationally standardized post-training assessments must be imposed, in Nepal as in every country that invests in SBAs. Secondly, the high proportion of SBAs conducting zero average monthly deliveries challenges the value of quotadriven targets for SBA training: instead of struggling furiously to churn out as many new SBAs as possible, countries and their development partners should think instead about redesigning trainee selection, such that those who are already engaged in clinical care-like midwives and obstetric nurses-receive training, whereas those who are engaged in community work-like public health nurses-do not. Finally, the low numbers of deliveries conducted by formally trained SBAs begs the question: who is carrying out most of the deliveries in these settings? Should the focus of our training be on them? An estimated 23-43% of births in Nepal are attended by traditional birth attendants (TBAs). While the WHO has specifically excluded TBAs from their recommendation for skilled attendance at birth, they can play a valuable role where skilled care is available but underutilized, by serving as liaisons between rural women and health providers due to their existing social and cultural importance [23]. Training TBAs to provide prenatal and postpartum care, to recognize warning signs of pregnancy complications during delivery, and when to refer a woman to a health provider has the potential to improve maternal and newborn health outcomes in Nepal [24]. Of those SBAs in our sample who did attend births, the vast majority oversaw normal deliveries, with fewer than 15 percent managing complicated deliveries. This problem was particularly stark at the lower-level facilities, where three-quarters of SBAs who managed an average of zero complications were based. Possible reasons for this are manifold. As previous studies have reported, pregnant women might be choosing to bypass lower-level health facilities for hospitals, to seek more advanced obstetric care [25,26]. Lower-level facilities might also offer fewer resources on average for managing complications, or staff less experienced SBAs. Both of these possibilities are supported by our findings: SBAs at lower-level facilities had accrued fewer months of nursing and SBA experience than their hospital counterparts. And lowerlevel facilities posted categorically poorer enabling environment scores than hospitals, reflecting at least partly a lower stock of basic emergency obstetric equipment that are integral to managing complicated deliveries. Interestingly, SBAs were split nearly evenly between permanent and temporary nurses-the latter of whom have acquired particular importance in rural Nepal, where the government has funded their recruitment to improve access to maternal health services [22]. Temporary nurses were found to be more productive than permanent ones; therefore, staffing these facilities with an even greater proportion of contract nurses might boost volume of both normal and complicated deliveries. Additionally, the vast majority of SBAs in our study were ANMs (91%), which in Nepal are maternal and child health workers, critical in rural areas, who have undergone a minimum of 18 months of training. The remaining 9% SBAs were staff nurses, who must undergo at least three years of training to receive their Proficiency Certificate Level in Nursing [27]. The lower level of training ANMs undergo compared to staff nurses and doctors could in part explain their poorer performance on the knowledge and clinical skills portions of the FEP.
To be effective, future SBA training policies should more carefully consider these inequalities, and design programs that more closely reflect the needs and constraints of different sites of work. Key to this effort will be equipping training programs with sufficient financial, technical and human resources to conduct monitoring and follow-up. Also important will be stocking health facilities with the equipment and basic amenities, like water and electricity that allow SBAs to fully exercise their skills: across our sample, SBAs ensconced in a positive enabling environment demonstrated greater mastery of clinical skills and a deeper fund of knowledge than those who were not. This leads us to conclude, if SBAs are supported from training through practice by a positive enabling environment, they are more likely to demonstrate sustained competence and productivity in the field.

Limitations of this study
Though we assessed a sample of 511 SBAs in Nepal, we did not directly administer their training and we utilized secondary data which was not solely for the purpose of research. With an observational, descriptive study, we cannot ascribe causality. Moreover, we examined SBAs solely in Nepal; and though Nepal has historically served as a bellwether country for maternal health goals, it is possible that some of our results are idiosyncratic to its history and training program approach. As for our report of the enabling environment indicators, we are aware that the availability of a structural indicator, such as a partograph, does not necessarily translate into its proper use; therefore, high scores for some indicators might mask wide variations in clinical utility. Importantly, however, the enabling environment score is intended to be more a heuristic and foundational tool for understanding the resource limitations of facilities. From a policy perspective, accordingly, this should not alter the validity of our findings. Finally, maintaining internal and external validity is a challenge due to potential error or bias introduced in the sampling, measurement, data collection, and data analysis phases of the study.

Conclusion
In spite of consistent and ongoing investments in the training of skilled birth attendants, in Nepal as around the world, our study suggests that these healthcare workers may not receive effective training, sufficient practice, or the appropriate experience to stay clinically competent and knowledgeable. The root of this problem seems to lie not merely in the attrition of skills over time, but also and fundamentally with the training programs themselves. Though many questions remain-for instance, under what circumstances a continuing education program might mitigate these outcomes-our study raises an important need to re-examine SBA training policies that advocate scale without evaluating quality. And it reminds us to relentlessly demand accountability, for both SBAs and the families they serve.
Supporting information S1