https://orcid.org/0000-0001-8784-4983
Correction
25 Mar 2021: Legesse S, Alemu T, Tassew M, Shiferaw B, Amare S, et al. (2021) Correction: Evaluation of in-service training program of laboratory professionals in Amhara Public Health Institute Dessie Branch, northeast Ethiopia: A concurrent mixed-method study. PLOS ONE 16(3): e0249429. https://doi.org/10.1371/journal.pone.0249429 View correction
Figures
Abstract
Background
In-service training programs should be evaluated and modified regularly to enhance training quality. However, in Ethiopia, there is no published evidence regarding its effectiveness. Therefore, we evaluated the Amhara Public Health Institute Dessie Branch (APHI_DB) in-service training program using the Kirkpatrick model.
Methods
In October 2019, a concurrent nested mixed method facility-based cross-sectional study was conducted among 107 laboratory stakeholders from 22 randomly selected government health facilities in the eastern part of the Amhara region. The qualitative part involved interviews with each of these key stakeholders. We collected data using a semi-structured questionnaire through face-to-face interviews. EpiData 3.1 and Microsoft Excel 2016 software were used for data entry and analysis respectively. The major qualitative findings were narrated and summarized based on four thematic areas to supplement the quantitative findings.
Results
A total of 107 laboratory personnel were interviewed, which makes a response rate of 97.3%. At the reaction level, 82.1% of the participants agree/strongly agree with the course structures, training contents, and learning tools. Likewise, 85.4% of the participants agreed/ strongly agreed on the trainer’s knowledge and their communication skills. In addition, 93.1% of the participants stated an improvement in knowledge and skills after attending the training. As a result, 65.6% of them were able to transfer their knowledge and skills into practice. Regarding the training set-ups and environment, 45.1% of the respondents disagree/strongly disagree with the training hall, toilet, cafe, tea and snacks, financial process, and accommodation perdiem.
Conclusion
Generally, the laboratory in-service training program of APHI_DB was more or less effective. Our findings suggest regular monitoring of each training event and evaluation of training programs against a clearly defined criterion. Furthermore, the institute is mandated to create a conducive learning environment and well-established training setups for trainees.
Citation: Legesse S, Alemu T, Tassew M, Shiferaw B, Amare S, Tadesse Z, et al. (2020) Evaluation of in-service training program of laboratory professionals in Amhara Public Health Institute Dessie Branch, northeast Ethiopia: A concurrent mixed-method study. PLoS ONE 15(12): e0243141. https://doi.org/10.1371/journal.pone.0243141
Editor: Conor Gilligan, University of Newcastle, AUSTRALIA
Received: April 18, 2020; Accepted: November 17, 2020; Published: December 4, 2020
Copyright: © 2020 Legesse et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting information files.
Funding: We acknowledge Amhara Public Health Institute Dessie Branch for covering data enumeration expense. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: APHI_DB, Amhara Public Health Institute Dessie Branch
Introduction
The in-service training program is one of a commonly used strategy to improve health workers’ knowledge, skills, and behaviors on new health approaches and technologies [1, 2]. The ultimate aim of the program is to capacitate health workers with the ability to deliver quality and accessible health services for the entire population [3]. To achieve this objective, a periodic evaluation of a training program should be conducted to identify major program strengths and weaknesses [4]. This helps training organizers to learn from their previous experiences, to know which stages of the training were successful and which not, or whether the approach to the training should be changed or not [5, 6]. The training evaluation also allows the identification of factors that restrain or contribute to a better performance of trained health professionals, suiting the training actions to the organizational needs [7, 8]. After all, the results of the evaluation should be used to modify the contents and conduct of a training program [4].
In the Ethiopian healthcare setting, laboratory professionals perform routine and highly specialized tests to diagnose and/or aid in the treatment of diseases, preventing and solving problems with results, specimens, or instruments, and communicating test results to physicians and/or clinicians [9]. Therefore, they need to be trained on specialized instrumentation and techniques to analyze patients’ samples, such as blood, urine, stool, body fluids, tissues, and cells that might show cancer or other diseases [10]. Thus, the Amhara Public Health Institute Dessie Branch (APHI_DB) in collaboration with partners is providing a range of capacity-building training to laboratory and other health professionals in the eastern Amhara region. The training is of various types which primarily focuses on identified priority laboratory needs; which commonly includes specialized laboratory testing services like tuberculosis testing, HIV testing, malaria diagnosis, basic microbiology, viral load determination, gene expert, laboratory quality management system, strengthening laboratory management toward accreditation, safety and bio-security, antiretroviral treatment automation, sample referral linkage and internal audit.
These are independent training courses given for different training participants in separate training sessions. However, most trainees attend over one training type per year. Training contents and curriculum are developed nationally in collaboration with stakeholders and partners and delivered by the Amhara Public Health Institute Dessie Branch in-service training center. We recruit trainees who are working in the medical laboratory setting from public and private health facilities. Then, trainees take part in the short term, face-to-face and single training sessions for a maximum of ten days and return to their workplace. Theoretical sessions are supplemented with practical sessions, videos, case studies, and role-play to maximize a better understanding of the contents. However, in Ethiopia, though health care organizations including APHI_DB spend substantial amounts of time, effort, and funding on health professionals capacity building training, there is no published evidence on the effectiveness of in-service training programs of training centers. Therefore, evaluation of the APHI_DB in-service training program is found to be important, as it can provide information that would alert the plan for enhancing training effectiveness. In addition, it highlights the major strengths and weaknesses in the training stages and the trainer institute. Thus, we evaluated the overall effectiveness of the APHI_DB in-service training program based on a set of criteria common to the major training types listed above.
Methods of evaluation
Evaluation design and period
In October 2019, a concurrent nested mixed method, facility-based cross-sectional study was conducted to assess APHI_DB in-service training program effectiveness.
Study area and setting
The Amhara Public Health Institute Dessie Branch (APHI_DB) was established in 1982 as a regional laboratory research center and renovated as a branch public health institute in 2016. It is in Dessie town, eastern Amhara, 401 km far from Addis Ababa (the capital city of Ethiopia) to the northeast direction and 490 km from Bahir Dar (the capital city of Amhara regional state). The institute has three dominant entities; namely medical and public health laboratory, research and technology transfer, and public health emergency management [11].
The laboratory wing provides technical support to health facilities found in the six zones, provides and serves as an in-service training center for various capacity-building trainings, referral and specialized laboratory test sites, performs public health laboratories for epidemic-prone disease and conducts an external quality assessment, laboratory mentorship and accreditation. The institute has a mandate to establish and maintain a high-quality and sustainable laboratory system throughout the sub-region. It also delivers quality and accessible laboratory services related to the occurrence, causes, prevention and diagnosis of major diseases of public health importance and to establish and support National Laboratory Quality Assurance programs and systems [12].
To realize its mission, the institute benefits from the generous support of the Ethiopian and regional governments and various national and international organizations. Some organizations that support the institute include the World Health Organization through its Global Fund, Centre for Disease Control, Japan International Cooperation Agency, International Center for AIDS Care and Treatment Programs (I-CAP) and various non-governmental organizations through joint projects.
Sample size and sampling technique
First, out of 30 external quality assurance health facilities, 22 were randomly selected to represent the eastern part of the region. Then, nearly five laboratory personnel from each government health facility were randomly selected, making a final sample size of 107 respondents. The qualitative part also involved interviews with each of these key stakeholders. All participants were laboratory personnel with a minimum of one year of work experience and who had ever received training in Amhara Public Health Institute Dessie Branch within the past two years.
Data collection
A total of 107 laboratory stakeholders were interviewed in person at their respective health facilities using a semi-structured questionnaire. We also included all these stakeholders in the qualitative key informant interviews. The questionnaire had both open-ended and closed-ended questions. Respondent’s opinions, suggestions and comments were received through key stakeholder interviews. The questionnaire covered information on laboratory training profiles, trainee’s reactions towards the course structure, training content and training tools, trainer’s knowledge and communication skills, new knowledge and skills gained, change in job behavior and performance, training set-ups and environment. Our respondents were laboratory personnel from selected health centers and hospitals. The data were collected by the investigators, considering that they were not assigned as a trainer and not from laboratory directorate.
Data quality assurance
To minimize the subjectivity of responses from stakeholders, the investigators themselves took part as data collectors. During the session of each visit, we briefed the respondents about the purpose of the assessment, which was to evaluate the effectiveness of the in-service training program of APHI_DB; not merely the individual’s performances. Besides, respondents were asked to feel open and comfortable to give their opinions, suggestions and comments that can help to improve in-service training programs and future laboratory services.
Data analysis
We graded all responses as ‘strongly agree, agree, not to decide, disagree, strongly disagree and yes or no, except for the open-ended questions. Data completeness was checked daily and the quantitative data entry was done through EpiData 3.1 software and exported to Microsoft Excel 2016 for analysis. Data analysis was done at four levels, i.e. (i) Trainees reaction towards training, (ii) Trainers knowledge on the course & their communication skills, (iii) Learning of training contents, (iii) Changes in job behavior after attending training (iv) Training setups, benefits and financial issues. We computed descriptive statistics to describe the data. Then, we narrated the major qualitative findings and summarized based on the above four thematic areas to supplement the quantitative findings.
Ethical approval and consent to participate
Ethical approval was obtained from the Institutional Review Committee of Amhara Public Health Institute. A formal letter was given to each health facility to get permission and cooperation. After briefing the purpose of the study, oral consent was obtained from each participant.
Operational definitions
In-service training.
A range of short term, practical training courses given by APHI_DB for laboratory professionals working at health facilities to scaling up their knowledge and skills.
Training effectiveness.
We evaluated the overall effectiveness of APHI_DB in-service training program using criteria common to all training types (Table 2) given by the institute. For each level of evaluation, we judge effectiveness as not effective when average values fall less than 50th percentile, partially effective between 50th and 75th percentile and effective above 75th percentile. But, in Table 8, we used the term adequate, satisfactory and appropriate instead of the word effective.
Results
Socio-demographic characteristics of respondents
A total of 107 laboratory personnel who are working in 22 health facilities (18 hospitals and 4 health centers) were interviewed with a response rate of 97.3 percent. The proportion of male respondents was 60.7%. The age of the participants ranged from 20 to 49 years; 42% were between 25 and 29 years old. Most participants are laboratory technicians (53.3%), while 44% have Bachelor of Science degree in a medical laboratory. Regarding respondent’s work experience, 44% have one up to three years’ experience whereas 34.6% of them had over five years of work experience and the rest 21.5% are in between three and five-year experiences. The majority (84.1%) of the respondents are working in the hospital and the rest 15.9% are in health centers (Table 1).
Distribution of in-service training among respondents
As depicted in Table 2, about 53.3% of laboratory personnel were trained on the new national HIV testing algorism, 52.3% on tuberculosis fluorescent microscopy and Ziehl-Neelsen diagnosis, 69.2% on malaria diagnosis, 26.2% on gene expert and 69.2% on laboratory quality management system. Only 8.5%, and 6.5% of laboratory personnel were trained on dry blood spot/viral load sample collection and basic microbiology courses respectively.
The qualitative findings showed that training opportunities were almost equally spread across the health facilities and the number of trained personnel seems adequate. However, few of the respondents believed that there was trainee selection bias or selection criteria are not fair from facility, to facility and also the calling modality was not appropriate, particularly at the zonal health department level.
The satisfaction of trainees with the course structures, training contents and tools
In this theme, 65.4% of the study participants mentioned that training contents were in depth enough, whereas 29.9% had a reservation on this. Another 85% also stated that learning aids/training tools assisted their learning sessions. Most of the respondents (79.4%) agree/strongly agree that technology/lab equipment was working properly during the training time and able to provide all the practices for the course, though 17.8% of the respondents strongly disagree/disagree on this idea. The same proportions of respondents (80.4%) agree/strongly agree that the course provided opportunities to practice and would improve their job performance. Besides, most respondents (86.9%) agree/strongly agree that the outcome of the courses was successfully achieved compared to meet their general expectations of the training and the courses gave trainees a clear understanding of the goals and objectives of the training before they started the training (96.3%) (Table 3).
In agreement with the quantitative finding, most of the respondents in the qualitative in-depth interview believed that the course structure, content and training tools of the training were clear and understandable; courses providing opportunities to practice and the contents were in-depth enough. Similarly, the availability of laboratory equipment and tools has been appreciated, especially in Strengthening Laboratory Management Towards Accreditation and Automation training. Some of the respondents risen “too short practice sessions and shortage of necessary machines and equipment" as a problem regarding course structure, contents and training tools.
Trainer’s knowledge and skills during training the course
The proportions of respondents that strongly agree/agree on the trainer’s knowledge about the course contents, their communication and delivery skills were nearly similar. Hence, 91% of respondents strongly agree or agree that trainers be able to deliver training courses properly and most respondents (88.8%) strongly agree or agree that trainer’s communication skills were able to provide all the information needed for the course. About 86% of respondents strongly agree or agree regarding the trainer’s knowledge of the course. Yet, 13.7% of respondents strongly disagree or disagree with that of the trainer’s ability to create a participatory and interactive training environment; though 12% of the respondents cannot decide on this issue (Table 4).
Findings from the qualitative part also revealed that most respondents acknowledged the trainer’s knowledge, communication and delivery skills. Others also reported more positive notions of knowledge, skills, and willingness to train based on their praising. In contrary to these, as stated by a few of the respondents, some problems related to knowledge and skills of trainers includes; undermining of a trainee, a few trainers are aggressive, sometimes trainers are none expertise or non-laboratory professionals, few trainers presentation styles is lecture so better to be entertaining style, trainers did not take the training of trainers.
New knowledge and skills gained in the training (learning the contents)
In this stage, 93.1% of the trainees strongly agree/agree that their knowledge and skills have improved because of having attended the training. Likewise, 92.5% of them strongly agree/agree that the practical sessions of the training have improved their skills and professional competencies (Table 5).
Change in job behavior and performance in the workplace (behavioral level)
At this level of evaluation, 93.5% of trainees mentioned as if they are applying the knowledge and skills they gained during the training in their workplace, and 69.2% of them tried to qualify other laboratory personnel’s in their working areas, 68.2% of them are using the training materials properly. However, 73.8% of the respondents replied no registration system that shows trained staff in the lab unit of their facility, and 55.1% of them also showed that they didn’t assign trained persons to the appropriate positions. On average, as stated in Table 6 below, 65.6% of the trainee’s transferred their knowledge and skill they gained during the training into practice.
Training setups and environment
In this theme, more than half of the respondents (51, 4%) disagree/strongly disagree with the suitability of training setups and environment (training hall, toilet, and cafe) for training. In the meantime, 46.7% of respondents strongly disagree/disagree with the accommodation perdeim and/or refreshments of the training program. Also, 28.9% of the respondents mentioned as if the calling modality for the training was inappropriate. Still, 69.2% of the respondents strongly agree or agree on the time and season of the training (Table 7).
The finding from the qualitative part also revealed that many of the respondents had a complaint on training setups, benefits and financing processes. Thus many of them have been fill discomfort with the training hall, toilet, cafe, the financing process, insufficiency of perdiem, and not happy with tea and snacks. On top of these, some respondents have been disgusted by the extended waiting time, guilty words and insults to receive their accommodation perdiem.
Overall, the authors have tried to summarize the effectiveness status of each level of training evaluations. As a result, participant’s reaction and their learning events were rated as satisfactory and adequate, respectively; while post-training behavioral change and the training setups were less satisfactory and inappropriate respectively. In general, the average cumulative scores of each level of evaluations fall in the fourth quartile (76.2%). Thus, as per our operational definition, this figure shows as the in-service training program of APHI_DB was more or less effective (Table 8).
Discussion
This study has tried to assess the effectiveness of the Amhara Public Health Institute Dessie Branch in-service training program using the Kirkpatrick models of training evaluation. Thus, in the first theme of evaluation, most participants were motivated and reacted positively to the course structures, contents and training tools which shows excellences in pre-training phase like training module preparation, method selection, training need assessment and trainee recruitment. Similar finding is also documented in a study from Egypt [13].
Regarding the trainer’s knowledge and skills on the course content, most participants point out that the trainer’s knowledge about the course content, their good communication and delivery skills were adequate to provide all the information needed for the course. This shows the availability of an experienced trainer that has received “training of trainers” on each training type. Besides, allocating enough trainers with sufficient training time will enable trainers to entertain novel ideas in a room which provides more discussion opportunities to trainees to grasp all the knowledge and skill competencies of the training and in depth exchange of best practices and challenges. Conversely, the allocation of non-competent trainers in the present study could be attributed to trainer’s knowledge gap and/or lack of expertise or intentionally assigned by training organizers; just for the seek of individual benefits or because of lack of experienced trainers. These finding shows a weakness in providing training to address actual training gaps. This kind of problem could be resolved by allocating most senior and experienced trainers rather than the mere allocation of trainers based on some other criteria.
In the second level of our evaluation, most participants mentioned that they have refreshed and/or gained new knowledge and skills during the training. In agreement with this, a study done in Sweden and Sri Lankan also pointed out the effectiveness of in-service training to equip trainees with the new knowledge and skills required for the job [8, 14]. Another systematic review emphasized the importance of training to health professionals to enrich their pre-existing knowledge and skills [15]. Our finding at this level implies the effectiveness of short term in-service training to strengthen health workers’ capacity on new health approaches or technology for improved health service provisions. This finding for in-service training program holders suggests the importance of improving in-service training coordination, planning, curriculum development, provision, evaluation and accreditation.
Our evaluation at the behavioral level asserted that the knowledge and skills they gained during the training have positively affected their work performance in most of the cases. They also evidenced this result in a recent systematic review of essential newborn care training that suggests improvements in delivery room newborn care practices as compared with usual care [16]. In this phase, our finding implies the success of APHI_DB training program to achieve its goal deeming that success at a behavioral level will lead to a result. Yet, trainee’s implementation effort and their competences should be further strengthened through supportive supervision to facilitate the outcome of the training.
Nearly half of the trainees declared that the training setups and environments were not suitable and appropriate for learning. Thus, many of them have been fill discomfort with the training halls, toilets, cafe, the financing process and insufficiency of perdiem, and not happy with tea and snack refreshments. These could be partially explained because the institute is currently working in temporary and non-standardized building blocks, with confined and limited classrooms. Besides, the daily perdiem might not be enough to accommodate trainee’s daily expenses in Dessie city, which disturbs trainee’s attention and mind during their stay in the training. Unless otherwise these kinds of problems are identified and resolved early, they will grow and amplify themselves to the extent of causing a significant negative impact on the training program and organizational achievement [4].
A limitation of the present study includes the lack of an observational part of the trainee’s practice and performance in their workplace that usually affects the quality of laboratory tests. Trainees’ knowledge and skill improvement are measured subjectively based on trainees’ responses without pre and post-test results. Also, this study did not address the result level of Kirkpatrick model that investigates the total training costs against its benefits. This shows the need to study further at a deeper level to get a holistic picture of APHI_DB training program effectiveness.
Conclusions
Generally, the laboratory in-service training program in Amhara Public Health Institute Dessie Branch was effective with its limitations. To meet its overall organizational aim, the institute should regularly monitor and evaluate the effectiveness of its in-service training program based on a pre-defined set of performance criteria. Training organizers should use lessons learned /feedbacks in the present study to create a conducive training environment for learning which improves the quality of training services.
Supporting information
S1 Appendix. This is the tool used for data collection.
https://doi.org/10.1371/journal.pone.0243141.s001
(DOCX)
Acknowledgments
We acknowledge Amhara Public Health Institute Dessie Branch for its support in conducting this study. We also appreciate the study participants for their willingness to take part in the study.
References
- 1.
USAID ASSIST Project. A global improvement framework for health worker in-service training [Internet]. 2015. https://www.usaidassist.org
- 2.
World Health Organization. Transforming and scaling up health professionals’ education and training: World Health Organization guidelines 2013.
- 3.
Ethiopian Ministry Of Health. Health Sector Transformation Plan. 2015.
- 4.
International Atomic Energy Agency. Means of evaluating and improving the effectiveness of training of nuclear power plant personnel. 2003;
- 5. Alvarez Kaye, Salas Eduardo G CM. An Integrated Model of Training Evaluation and Effectiveness. Hum Resour Dev Rev. 2004;3: 385–416.
- 6.
Ministry of Interior and Japan International Cooperation Agency. Manual on Training Evaluation.
- 7.
Kebede F, Asres A, Gutema K, Wuliji T. 2012. Towards a National In-service Training Strategy. Workshop Report. Published by the USAID Health Care Improvement Project.
- 8. Swedberg RN Lena, Michelsen Hans, Chiriac Eva Hammar H I. On-the-job training makes the difference: healthcare assistants’ perceived competence and responsibility in the care of patients with home mechanical ventilation. 2015; pmid:25181407
- 9.
Ethiopian Standards Agency. Advanced Medical Laboratory Requirements. 2012;
- 10.
The Lewin Group. Laboratory Medicine: A National Status Report. 2008;
- 11.
Council of the Amhara regional state. Amhara Public Health Institute establishment, proclamation No. 138/2016. 2016.
- 12.
Amhara Public Health Institute. Medical laboratory departement working guideline (unpublished work). 2018.
- 13. Khalil Sally. The Evaluation of DEMEC Training Program Effectiveness by Using The eQvet-us Model. 2017; 182–198.
- 14. Rajapakse BN, Neeman T D A. The Effectiveness of a ‘Train the Trainer’ Model of Resuscitation Education for Rural Peripheral Hospital Doctors in Sri Lanka. PLoS One. 2013;8: 1–16. pmid:24255702
- 15. Abdel-All M, Putica B, Praveen D et al. Effectiveness of community health worker training programmes for cardiovascular disease management in low-income and middle-income countries: A systematic review. BMJ Open. 2017;7: 1–11. pmid:29101131
- 16. Opiyo N, English M. In-service training for health professionals to improve care of seriously ill newborns and children in low-income countries. Cochrane Database Syst Rev. 2015;2015. pmid:25968066