The HMS BAB training

The HMS BAB training intervention was provided to all maternity health workers using the 2013 basic HMS BAB curriculum developed by Laerdal Global Health and Jhpiego [21, 22] as part of our HMS BAB trial [13, 28].

First Jhpiego HMS BAB master trainers trained 12 district-HMS BAB trainers during a one-week training in a central location and later travelled to the districts where the HMS BAB in-facility trainings were conducted [13, 28]. One-day-long in-facility training were provided to all available health workers in maternity units, following same schedule and content. In each facility, district trainers selected two health workers using a specific criterion to become “peer practice facilitators”. These peer practice facilitators were further trained to lead eight-weekly so called “practice drills” that were mandatory sessions within their facilities. These “practice drills” were based on clinical scenarios and were to be conducted for 30–40 minutes per week. This training technique is referred to as “low dose high frequency” as learners are exposed to frequent repetitive learning exercises. Health workers who performed poorly during the sessions were given more practice until they mastered the specific simulated skill. Participants received refreshments during the in-facility training sessions. All facilities received Mama Natalie training kits for use during these sessions.

Participants

In this secondary data analysis, we included all the trained maternity ward health workers of participating health facilities: 636 health workers were trained. 337 in January 2016 in 31 intervention facilities of HMS BAB trial and 299 in January 2017 in 30 comparison facilities after the end of the trial. The 10-month assessment in a subset of health workers were done only in intervention facilities for operational reasons.

Assessments and tools

We used the standard Jhpiego pre- and post-training assessments [21, 22] which comprised a written knowledge test with 15 points multiple choice and True/False questions. The tests were made available in the local (Swahili) language and were self-administered. The written test was followed by simulation session where each participant was asked to manage a simulated case of PPH including recognition of retained placenta practice of simulated skills to prevent, detect and do basic management of PPH. Each theoretical session was followed by a specific pre-training assessment, demonstration followed by discussions and later post-training assessment of the skill. Skill assessments were done by validated Objective Structured Clinical Examinations (OSCEs) checklists using three simulate scenarios: performing Active Management of Third Stage of Labour (AMTSL), managing retained placenta and managing severe PPH secondary to uterine atony [22, 23]. For a subset of health workers re-assessment using same tools was performed at 10-month follow-up.

Additionally, the participants’ professional qualification, years of professional experience working in the maternity ward, number of deliveries assisted in the last 30 days, and whether they had ever received previous in-service training in AMTSL or PPH was documented.

Data handling and analysis

Data were manually entered in MS excel and checked for completeness. To analyse the data, we created a new score variable combining the three OSCE skill scores into a mean skill score. Analysis was done by STATA software version 15. Tables of summary statistics were prepared to describe the data: mean or median and standard deviation or Interquartile Range (IQR) were used to summarize continuous variable, frequencies and percentages to summarize categorical variables. Years of professional experience was categorized into <2 year, 2–4 years, 5–7 years, and ≥ 8 years. Number of assisted deliveries in the last one-month was categorized in ≤5 deliveries, 6–15 deliveries, and ≥16 deliveries assisted in the last one-month. Professional qualifications were categorized into (1) medical doctors (2) other clinicians (assistant medical officers and clinical officers); (3) nurse-midwives included all nurse-midwives with diploma and above; (4) auxiliary staff and (5) others included laboratory technicians. The facility level was categorized into hospitals, larger health centres that performed Caesarean sections (C-sections) and lower health centre without C-sections. Previous in-service training (ever) of AMTSL or PPH was set as a binary variable.

The outcome variables were absolute scores and mean change of scores in knowledge and aggregate OSCE skills. Aggregate OSCE skill score was an average of all three OSCE scores: AMTSL, simulated retained placenta and severe PPH due to atony and is referred as skill scores throughout. Immediate change of scores was calculated as the difference between immediate post-training score and pre-training score and change at follow-up (decline) was calculated as difference between scores at follow-up and that of immediate post-training All scores were summarized and mean with 95% confidence intervals presented in tables. We compared individual health workers’ knowledge and skill scores to assess change immediately and that at follow-up period using paired t-test and reported significance results at p-value <0.05.

We used linear regression analysis to assess the predictors of change of knowledge and skill scores 1) immediately after the training and 2) at 10-month follow-up, for each of the independent variables and present coefficients with 95% confidence intervals. Significance level was set at p<0.05.

Ethics and consent

Ethical clearance was granted from Muhimbili University of Health and Allied Sciences IRB with permit number 2015-11-18/AEC/Vol X/71 and 2016-12-14/AEC/Vol XIII/03. All participants were informed about the study and written informed consent sought and obtained. The benefit of participating in the study was improving knowledge and skills that lead to better performance and job satisfaction. There was no potential harm from participating in the training, except for the inconvenience of assessments. All participants were given codes to ensure follow up of scores without identifying individual performance. All personal identifiers were removed from electronic data.

Main findings

Our study indicated significant increase of health workers knowledge from 74.2% (95% CI 72.8, 75.5) to 89.2% (95% CI 87.5, 90.9) and skills from 38.2% (95% CI 36.8, 39.60) to 85.4% (95% CI 84.3, 86.5), a 47.8 percentage-point increase (95% CI 46.5, 49.2) immediately-after the training. Pre-training skill scores were lower than knowledge scores. Auxiliary staff showed a larger increase in scores from the training (49-percentage-point, 95% CI 47.1, 52.8) compared to other professions, but the 10-month follow-up assessment also indicated that this group had the largest decline of -11.4% (95% CI 21.2, -1.6).

Profession, number of deliveries assisted in the last month and years of professional experience were associated with skill change immediately post-training. Health workers assisting 6–15 deliveries in the last month gained 2.4 percentage-points more than those assisting <5 deliveries (p-value 0.021) immediately post-training and displayed less skill decline at 10-month follow-up. Health workers with more than 8 years of profession experience significantly scored less than those with <2 years of experience. Working in health centres that performed C-section and ever attending prior in-service PPH training (but not AMTSL) were associated with higher gain in skill scores.

Our study supports the existing evidence that the HMS BAB one-day training with mandatory weekly practice sessions is beneficial to upgrade health workers’ knowledge and skills in prevention, assessment and basic management of PPH [15, 22, 23]. Like other studies the median gain in skills was higher than for knowledge [20], perhaps due to less chance to improve the already high knowledge scores.

We observed the largest gain in skills score among auxiliary staff. Although auxiliary staff is not officially counted as ‘skilled birth attendants’, they assist deliveries as part of their routine care in the studied facilities. A recent publication by Rao et al. (2018) from India proposed that lower cadre health professionals that are not trained for delivery care can be trained and mentored to perform routine delivery care and manage PPH to the same level as nurse-midwives [31]. While this is an encouraging finding, our study indicated lower pre-training knowledge and skills as well as lower retention of knowledge and skills over time compared with nursing and clinical professions. Thus, the employment of auxiliary staff for childbirth care needs to be considered with caution as larger efforts and investments are needed to support them to maintain a minimum knowledge and skills level. At the same time, it is important to recognise that auxiliary cadres provide extra hands on the ground in a resource constrained setting with a limited workforce. Only 12–27% of auxiliary staff had ever attended in-service training in this study, as in Tanzania they are usually excluded from these training as they are not considered skilled. We believe they should be provided with such trainings so that they also know what needs to be done in case of emergencies.

Our study concur with similar studies of in-service trainings that reported being active in clinical duties in the maternity ward was associated with better skills and also more retention of acquired skills following a training [12, 19, 22]. To date there is no consensus as to what an acceptable number of deliveries in a month is that a health worker can manage, strengthen and maintain the learned skills without burn-out. In this study, health workers who self-reported to have ever attended a prior PPH in-service training had higher scores than those not attended. This was not observed for AMTSL in-service training. We could not explain this, however, we did not assess contributing factors like time that elapsed from the reported training or training modality. Such factors need to be explored in future studies.

Studies on knowledge and skills retention reported different results on retention depending on time from initial in-service training from 3–24 months [19, 20, 32]. Retention of skills was reported to be better in settings where there are regular local mentoring programs, refresher trainings or repeated assessments [20, 33, 34] and continuous skills drills like the ‘low dose high frequency’ approach used in HMS BAB [15, 20, 32]. The HMS BAB training emphasized simulated practice, repetition and discussions and was specifically developed to focus on more practical training [35]. In this study, all health facilities had a mandatory weekly scenario-based practice session for 8 weeks following the initial training, led by local peers that could have led to the limited observed decline in knowledge and skills scores. Other factors that have been found associated with a skill decline in similar training modules were lower level facilities [20], prior in-service training and years of professional experience [35].

We were only able to include 63.7% of health workers in our 10-month assessment due to a variety of reasons illustrated in Fig 1. Only 9.5% of staff were transferred out of maternity wards to other departments which is an encouraging finding. High turnover and rotation of health workers from maternity wards is a common practice in the country, however not much has been documented about its potential for negative impact on individual health workers’ knowledge and skill retention and practice, team response when managing complications such as PPH and maternal health outcomes.

Methodological consideration

The strength of our study is the large number of participants which allowed us to disaggregate the effects of the training in several sub-groups. We included 636 participants from 61 facilities: 605 had complete immediate post/pre-training knowledge scores and 591 complete immediate post/pre-training skills scores. Previous studies had smaller sample size of between 30 to 145 health workers [22, 24].

Another strength is that we used OSCE assessment for observation of simulated skills and used the same assessment tools during all the assessments periods, hence improve comparability. We report on aggregate skill score and change for three core competences at 10-month follow-up: rather than single skill scores as we were interested in a single score that may represent prevention and basic management of PPH portraying a more comprehensive score.

However, our study has limitations. We chose the 10-month follow-up for reassessing knowledge and skills retention, for pragmatic reasons, as we provided at this time period the training to the comparison facilities after the finalisation of the HMS BAB trial. While other authors did follow-up at several points, this was financially not possible for this study. Another limitation is that we missed fourteen health workers due to poor road infrastructures. We only reassessed 63.7% of the eligible number of participants, the remainder were absent or refused. We do not know whether those who refused or could not participate for other reasons had poorer or better performance and how this would have affected the results. However, the absentees were mainly medical doctors (3), other clinicians (14) and nurse-midwives (68), we hypothesize that perhaps their absence should not have affected overall results. Furthermore, our sample had few medical doctors at the initial training. We were only able to include two doctors and eleven clinicians in the 10-month assessment. Seven of thirteen medical doctors included in the training were interns and had moved to other employment areas. This limits the generalisability of our findings in relation to clinical professions. We also acknowledge that using same assessment tools for before and after assessment could mislead as health workers may be familiar with the questions and could have studied them to pass.

As part of the overall HMS BAB trial, health workers were to perform weekly practice drills for 8 weeks. We assumed all those trained in January 2016 had an opportunity to have done the practice drills as outlined by the intervention prior to the 10-month follow-up however, this may not be true. Twenty-three of 31 facilities trained in January 2016 completed the weekly practice drills within three months, three facilities did five sessions and five facilities did two sessions at this time. Although there were logbooks that showed participation of health workers during weekly drills, it was not possible to link individual providers to the number of drills attended and the decline of knowledge and skills. Nonetheless, a recent publication by William et al (2019) utilizing similar training practice drills reported less than one practice session per 8-weeks per health worker [36].

We report here on test scores. While preferably we would have reported on improved clinical practice, such assessments are logistically and practically difficult. However, we believe that our findings reflect training outcomes of improved test scores and retention was also reflected in the findings of the larger HMS BAB trial [13].

For some categories, for example the number of deliveries assisted in the last month or years of professional experience We have used cut-off points informed by mean and median exploration of the data for some categories. There are no clear standards and different authors have used different cut-off points which reduces comparability [12, 35, 37]. This may differ depending on the context, but there need to be some framework to inform on acceptable number of deliveries assisted in the last month per health worker [38].

Implication for practice

Our study illustrates the potential of the HMS BAB training to improve knowledge and skills of health workers in maternity wards. While all health professionals benefited from the training, auxiliary staff benefited most, but they also had the lowest skills retention. This finding cautions the integration of lower trained staff into service delivery and a careful context specific assessment of advantages and disadvantages needs to be done if such professionals are used to fill shortages of qualified staff.

In-service training is widely used to improve health worker performance and the relevance has been underscored by systematic reviews [8, 9]. The HMS BAB training, in contrast to other trainings which are offered outside the facility premises, includes all health workers regardless of their professional background. We believe this is a strength of the training and our results underline the relevance for all professions. However, our study provides information on the difficulties to train all maternity health workers and to create stable teams. While twenty-five health workers dropped out early, others did not finish all the training components of the training as they were called to attend patients. About one-third of trained health workers were absent at 10-month follow-up because of various reasons. Training and other quality improvement initiatives need to be sensitive to such attrition and one-off interventions are likely to be insufficient.

Our study highlights gaps around understanding health workers characteristics that can strengthen training and use of resource.