Evaluation of the performance of health extension workers on malaria rapid diagnostic tests and predictor factors in Bahir Dar Zuria district, northwest Ethiopia: A cross-sectional study

Background In Ethiopia, anti-malaria treatment is initiated after parasitological confirmation using blood film microscopy at health centers and hospitals, or serological rapid diagnostic tests at health posts. At health posts, the diagnosis is performed by health extension workers using rapid diagnostic tests after little training. However, there is paucity of data about the health extension workers’ performance on rapid diagnostic tests. Hence, periodic monitoring of the performances of health extension workers on malaria rapid diagnostic tests and predicted factors plays a pivotal role for the control of malaria. Methods A cross sectional study was conducted in May 2020, among 75 health extension workers working at health posts in Bahir Dar Zuria district, Northwest Ethiopia. Their performance on malaria rapid diagnostic tests was assessed by distributing known positive and negative samples as confirmed by investigators using both rapid diagnostic test and blood film microscopy. Test results from health extension workers were then compared with that of investigators. Procedural errors committed while performing the tests were assessed using observational checklist. Data were analyzed using SPSS software version 20. Results The overall sensitivity and specificity of health extension workers in detecting Plasmodium species were 96.8% and 98.7%, respectively with 97.3% result agreement between the health extension workers and investigators (kappa value = 0.949). The most common procedural errors committed by health extension workers was ‘not checking expiry date of the test kits’ followed by ‘not adhering to the appropriate time of reading results’ that 70.7% and 64% of the participants committed these errors, respectively. Total number of procedural errors committed by those who have got in-service training was decreased by 47.3% as compared to those without in-service training. Conclusions Health extension workers had high performance on malaria rapid diagnostic tests. However, in-service training and periodic supervision should be given in order to maximize performance on these tests.

Extension Program (HEP) in Ethiopia was initiated in 2003 with the main objectives of prevention and control of communicable diseases like malaria [12].
The HEP is implemented at health post or community level which is the lowest level of the Ethiopian health tier system. The HEWs are community health workers who are trained for one year in order to provide basic health services such as family planning, vaccination, health education, diagnosis and treatment of certain diseases like diarrhea and malaria. In this program, two female HEWs are assigned to a health post in each Kebele (village). Health extension workers are recruited from the communities in which they will work according to specific criteria: they are female (except in pastoralist areas), at least 18 years old, have completed at least grade 10 in education, and speak the local language. Females are selected because most of the HEP packages are related to issues affecting mothers and children; since communication is thought to be easier between mothers and female HEWs [13].
The main objective of the HEP in Ethiopia is to improve access to essential health services provided at the village and household levels [13]. Despite the main task of HEWs is to work on disease prevention, they are also involved in diagnosis and treatment of common diseases like malaria. As a result, at health posts where there is no laboratory and laboratory professionals, HEWs are given little training and screen febrile cases with RDTs before prescribing antimalaria drugs. They treat patients with uncomplicated malaria using artemisinin combination therapy, chloroquine, and primaquine based on the national malaria diagnosis and treatment guideline [14].
Despite malaria RDTs are easy to perform with little training, strictly following the manufacturer's instructions is important to have reliable results. Hence, periodic monitoring of HEWs while performing the RDT procedures is mandatory to take timely corrections, in case errors are committed. However, there is paucity of data about the HEWs' performance on malaria RDTs in Ethiopia. Therefore, the present study aimed to assess the performances of HEWs on malaria RDTs and predicting factors in Bahir Dar Zuria district, where malaria is a public health problem.

Study design, area and period
A cross-sectional study was conducted at all health posts in Bahir Dar Zuria district, Northwest Ethiopia in May 2020. Bahir Dar Zuria district is one of the 14 districts in West Gojjam Zone, which is located at a distance of 560 km from capital city of the country (Addis Ababa) and it is situated surrounding Bahir Dar city, capital city of Amhara National Regional State. The altitude of the district ranges from 1700-2300 meters above sea level [15]. The district has an average annual rainfall of 1035 mm. The minimum and maximum temperature lies at 10˚C and 32˚C, respectively [15]. Likewise, the major transmission season of malaria occurs from September to December followed by April to May with minor transmission [14]. The district has 36 kebeles and 9 health centers and 36 health posts where 81 HEWs have been employed.
Study participants' recruitment. All HEWs who were actively on duty during data collection period and gave consent to participate in the study were included.

Data collection
Data on predicting factors for committing procedural errors while performing RDTs were collected using structured questionnaire. The questionnaire was designed to collect information about HEWs' age (in years), prior experiences in performing malaria RDTs (in years), prior in-service refreshment training on malaria RDTs (yes/no), supply of RDTs to the health post so that the service has not been interrupted (yes/no), frequency of supervisions per quarter year by the malaria officer or other responsible body assigned to support HEWs in the diagnosis and treatment of malaria (number of visits per every 3 months), trust in RDT result by HEWs or their feelings in the accuracy of RDT results (trust/not trust).
The HEWs' performances on RDTs were evaluated using known positive and negative samples as confirmed by investigators using both RDT and blood film microscopy. Two ml of venous blood was collected by the principal investigator from laboratory confirmed malaria cases for P. falciparum (pf), P. vivax (PV) and P. falciparum/P. vivax (pf/PV) mixed infections and from apparently healthy (non-malaria infected) individuals. Each of the collected blood samples was transferred into Ethylenediamine Tetra Acetic Acid containing test tubes. Seven blood samples (two P. falciparum, two P. vivax, one mixed, and two negative) confirmed by both microscopy and RDT were distributed to 34 health posts and HEWs tested all samples using their own RDTs. Then data on the performance of each procedural step were collected using observational checklist prepared for this purpose. The prepared checklists divided the RDT procedure into 8 steps in three domains: procedural (checking RDT expired date), safety (discarding the pipette in the sharps box), as well as accuracy (labeling of RDT, dispensing correct volume of blood, dispensing blood in correct well, dispensing correct volume of buffer, time of result reading, and correctly reporting the result). Among the seven samples to be run by each HEW, one RDT procedure (4 th sample) was strictly observed by principal investigator, noting whether the HEWs performed each step correctly or not.

Data analysis
Data were checked for completeness and entered and analyzed in statistical package for social science (SPSS) software version 20. Descriptive statistics like frequency and percentage were manipulated to explain the study participants and their test performance. Multivariable Poisson regression was fit to test for associations between error count data and prespecified covariates (age group, prior RDT experiences in years, prior in-service training on RDT, regular supply of RDTs to the health post, frequency of supervision per quarter year, trust in RDT result). Selected pre-specified covariates were included in the analysis based on the hypothesized relationship to the outcome. P-value < 0.05 in the multivariate analysis was considered as statistically significant at 95% confidence level. Kappa value was also calculated to show the strength of test agreement between 'HEWs and investigators' and interpreted as follow: values � 0 as indicating no agreement and 0.01-0.20 as none to slight, 0.21-0.40 as fair, 0.41-0.60 as moderate, 0.61-0.80 as substantial, and 0.81-1.00 as almost perfect agreement [16].
Performance of HEWs' on malaria detection and species identification using RDTs were tested by computing the sensitivity, specificity, positive predictive values (PPV) and negative predictive values (NPV) against investigators' results using Med-Calc software version 19.3. The software calculates the value for each parameter with the sense of the following equations. sensitivity = a/n1×100 specificity = b/n2×100 positive predictive value = a/n3×100 negative predictive value = b/n4×100 where n1 = number of true positives identified by investigator; n2 = number of true negatives confirmed by investigator; n3 = number of samples identified as positives by HEWs; n4 = number of samples identified as negatives by the HEWs; a = number of cases identified as positives by investigator and by the HEWs; b = number of samples identified as negatives by investigator and by the HEWs.

Ethical consideration
The research was carried out after ethical approval was obtained from the institutional review committee of College of Medicine and Health Sciences, Bahir Dar University with reference number CMHSc009/2020. Additionally, supportive letters were obtained from Amhara public health institute, West Gojjam Zone health department, and Bahir Dar Zuria district health office and permission was obtained from each health center authorities where clinical samples used to evaluate HEWs were collected. Information obtained at any course of the study was kept confidential. Informed verbal consent was obtained from participating HEWs and no personal and health facility identifier was included as participants were given a unique code. According to the research ethics guideline of Bahir Dar University, informed verbal consent is accepted to conduct researches where the data collection doesn't include invasive procedures (like aspirate or biopsy samples). Hence, the verbal consent for the present study was approved by the institutional review committee. Health extension workers who committed procedural errors while performing RDTs were given feedback on the spot about the procedural errors they committed and how those errors could be avoided.

Socio-demographic characteristics of study participants
Among the total of 81 HEWs employed in Bahir Dar zuria district, 75 participated in the study. The remaining 6 HEWs were not actively on work during data collection period due to different reasons. In this study, age of the participants ranged from 25-34 with mean age of 29.23 (± 2.077 SD) years. Forty-five (60.0%) participants have taken in-service training on malaria RDTs. The participants had work experience ranged from 5-14 years with mean experience of 8.96 (±2.5 SD) years ( Table 1).

Performance of health extension workers
From a total of 75 HEWs participated in the study, all the 8 procedural steps was carried out correctly by more than 70% of the participants, except for steps 1,5 and 7. The median number of total and accuracy steps completed correctly was 6 and 4, respectively. The number of errors observed ranged from 0 to 5 errors per HEW. The steps most commonly performed incorrectly were step 1 and 7 ( Table 2).

Interpretation of RDT results
A total of 525 (375 positive and 150 negative) samples were tested by HEWs. Of these, 511 (97.3%) were correctly reported. Among 375 positive samples, 363 (96.8%) were correctly reported as positive and the rest 9(2.4%) were falsely reported as negative. Out of 75 samples with mixed infection, 72(96.0%) were correctly reported as mixed and the remaining 3 (4.0%) were falsely reported as P. falciparum. From the 150 negative samples, 148(98.7%) were correctly reported as negative ( Table 3). The overall sensitivity and specificity of HEWs 'on detection of Plasmodium species were 96.8% and 98.7%, respectively while the overall positive and negative predictive values were 99.5% and 94.3%, respectively. The test agreement between the HEWs and investigators was 97.3% with kappa value of 0.949 (Table 4).

Poisson regression of factors associated with total error count
According to Poisson regression model, the outcome of total error count showed that in-service training on RDT was associated with the overall test performance. The mean error ratio (MER) for getting in-service training was 0.527(95% CI 0.365-0.763), which corresponds to the mean number of total errors for those who took in-service training being 47.3% (95% CI 23.7-63.5) lower when compared to those without training (Table 5).

Discussion
Malaria rapid diagnostic tests have been important alternatives to diagnose malaria at health posts in Ethiopia as the tests can be performed by HEWs with little training. However, monitoring of the RDTs performance of HEWs in conducting the tests is indispensable to maximize the diagnostic quality of RDTs. In the present study, the performance of HEWs on RDTs were evaluated and reported for the first time in Ethiopia as far as our knowledge is concerned. Health extension workers performed and interpreted RDT test results almost perfectly with sensitivity and specificity of 96.8% and 98.7%, respectively on detection and identification of Plasmodium species. The sensitivity of the present study was higher than the study reported in Western Kenya (92%). Such higher sensitivity in the present study might be due to variations in duration of experience in performing RDTs. However, the specificity of the present finding was comparable with previous result of 97.0% in Western Kenya [17].
As compared to the investigators' result, there were two negative samples reported as false positive (one P. falciparum and one P.vivax), nine positive samples as false negative, and three samples with mixed infection as P. falciparum mono infection. These false positive and false Table 4 negative results might be due to shipment and storage conditions of the RDT kits such as direct exposure to sunlight. Study conducted in Burkina Faso documented that transportation and storage condition of RDTs affect the test performance [18]. Moreover, false positive results might be due to delayed result reading. In the current study there were twelve observations where result readings were made after 20 minutes. In support of this, a study conducted in Damot Gale district, Southern Ethiopia found that HEWs believe that reactive results are visible after 20 minutes and they wait longer time to report reactive results [19]. The false negative results might be due to early result reading and dispensing inadequate volume of blood in addition to inappropriate storage conditions. Reporting of mixed infections as P. falciparum mono infections might be due to difficulty in the interpretation of faint positive test lines. Previous studies have also reported difficulties in the interpretation of faintpositive test lines by HEWs [20,21]. In our observation, there was also interpretation of two invalid results as negative (data not shown). A study from Zambia also reported that community health workers read faint positive and invalid results of RDTs as negative suggesting that subjective interpretation may contribute to false results [22]. These phenomena might be caused by excess amount of blood, prolonged exposure to hot temperature as well as transportation conditions of the RDT kits [23].

Parasite Sensitivity (%) Specificity (%) PPV (%) NPV (%)
The accuracy of RDTs can be affected by incorrect blood volumes, incorrect buffer volume and timing of the test interpretation. Excessive blood volumes can result in staining and obscuring test lines [24] and results in prozone effect [9]. This could cause a health worker to misread a positive result as negative, especially with a low parasite density infection where test lines tend to be faint. Inadequate blood volumes can also reduce test sensitivity by producing false negative results when antigen quantity is insufficient to generate a visible test line [24]. Excessive buffer volumes can also result false-positive results due to non-specific bindings. While, insufficient volume of buffer can lead invalid test result by impeding clearance of the strip and/or slow down migration with failure to generate a control line [25].
During our observation, we have noted that only two and four HEWs added too little and too much blood, and one and eleven HEWs added too little and too much buffer, respectively. This has a significant impact when viewed from the number of patients each HEW is serving every day.
Early and delayed reading of RDTs result was another common error observed in the present study as thirty-six and twelve HEWs have committed those errors, respectively. Previous studies have also noted that reading RDT results early is a frequently observed error [17,21,26]. Reading too early may cause false-negative results, while reading too late may cause falsepositive results due to a backflow phenomenon [25,27].
In-service training on RDTs was found to be significantly associated with lower procedural error rates. Getting training decreased the mean number of total procedural errors by 47.3% (95% CI 23.7-63.5) as compared to those without training. This finding is supported by previous studies showing that training on RDTs is associated with improved performance [17,18]. Moreover, another study underlines the need to provide at least minimal training prior to RDTs use, even for community health workers with prior healthcare experience [20].

Limitations of the study
Performance of HEWs in patient preparation and sample collection were not evaluated. The optimal condition of RDT such as storage conditions was not observed. Moreover, the presence of observers may have impacted HEWs' performance that formal observation can lead to a state of anxiety that decrease or increases in performance during supervision and evaluation [28,29].