Introduction

Globally, public health endeavours aimed at enhancing community well-being through health status evaluation, policy formulation, and service delivery assurance [1,2]. Given that, over 40% of countries lack evidence of adherence to data quality assurance processes in health facility data [3], the potential consequences of relying on unreliable data are severe [4].

The Health Management Information system (HMIS) collects, stores, analyses, and evaluates health-related data and is crucial for planning, monitoring, and evaluation of health programs and interventions [5]. The quality of data generated from HMIS is essential for effective decision-making in the health system [6].

Noting the crucial role of high-quality data, the World Health Organization (WHO) developed metrics for data quality [7]. The dimensions are completeness, consistency, and timeliness, there has been growing recognition of the importance of data quality in HMIS [8]. Completeness is the extent to which all necessary data elements are recorded and reported. Data completeness applies to various levels of reporting, from health facilities to districts and districts to regional levels. Consistency assesses the coherence and reliability of the data, both internally across different data sources (accuracy) and externally with other sources [9]. On the other hand, the timeliness of data measures whether the reports were submitted before the specified deadline [10]. It measures the promptness with which data are collected, entered the system, and made available for decision-making purposes.

Despite the pivotal role of the HMIS in Ethiopia’s healthcare system, challenges persist in ensuring data quality across different regions and health facilities. These challenges include human resource shortages, inadequate training in data management practices, infrastructural limitations, and behavioural factors that influence data reporting practices among healthcare personnel. Addressing these challenges is essential for enhancing the reliability, accuracy, and utility of HMIS data, thereby strengthening the evidence base for healthcare policies and interventions [11,12].

This systematic review aims to analyse data quality in the HMIS across Ethiopia, synthesizing findings from 2015 to identify trends, challenges, and best practices in terms of completeness, consistency, and timeliness. It provides strategies for enhancing HMIS data quality, improving healthcare delivery, and achieving better health outcomes.

Methods

Results

Search results

The initial search across databases identified 1790 papers, from which 122 duplicates were removed, leaving 1668 articles for screening. Following title and abstract evaluation, 1612 articles were excluded due to irrelevance, resulting in 56 articles for full-text assessment. Ultimately, 25 articles [15–39] met the inclusion criteria for the review. Exclusion reasons included lack of outcomes related to data quality dimensions (n = 20), focus on evaluating HMIS tool effectiveness (n = 5), inappropriate population (mostly related to medical records) (n = 4), study period mismatch (n = 1), and duplicate studies (conducted on the same population), Fig 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. The PRISMA flow chart for study selection and screening.

https://doi.org/10.1371/journal.pone.0316498.g001

Characteristics of the included studies

Eight studies (32%) were conducted either nationally or across multiple regions [19,26,28,29,33,34,36,38]. Specifically, four studies focused in the Amhara region [21,22,30,31], three studies in South Nations Nationalities and People’s Region (SNNPR) and Oromia region [16,27,39], two studies each for Addis Ababa [15,23] and Tigray [17,25]. Additionally, there were single studies from each region (Somali [35], Harari [18], and Benishangul Gumuz [32]). The studies were published between 2016 and 2024, reflecting recent assessments of HMIS data quality across Ethiopia’s diverse geographical and administrative contexts.

Most of the studies were cross-sectional studies, 11(44%) studies [15,21,25–28,30–32,34,37] used mixed design to incorporate qualitative findings as well. Almost all studies assessed maternal and child health indicators for assessment of data quality, two studies [18,23] failed to specify indicators used to determine data quality. Most included studies reported the following indicators: first antenatal visit (ANC1), fourth antenatal visit (ANC4), skilled birth attendance (SBA), postnatal care (PNC), child immunizations, and contraceptives. Few studies used the WHO tracers (indicators of data quality), and no study used all five indicators.

Most of the included studies compared the HMIS report and the registries (source document). Registers are records that are filled by health professionals at different units. Usually, these documents are filled out daily. Other studies compared the HMIS report with WHO/UN estimates of immunization [26], HMIS report with population-based surveys [16,19,33,36], the HMIS report with Emergency Obstetrics and Newborn Care Assessment data [38], and the HMIS report with JSI coverage data as well as linked to biomarkers using serosurvey for immunizations [29]. The characteristics of the included studies are summarized in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of included studies.

https://doi.org/10.1371/journal.pone.0316498.t001

Completeness of routine health data.

In nineteen studies (76%), completeness was the primary focus [15,16,18–25,27,28,31–36,39]. According to WHO guidelines [40], encompasses two dimensions: the completeness of reports reaching the next level and the completeness of indicator data. WHO-defined indicators include ANC first visit, 3rd dose of Diphtheria-Tetanus-Pertussis (DTP) containing vaccine, newly initiated antiretroviral therapy (ART), notified cases of all forms of tuberculosis (TB), and confirmed malaria cases [40]. Among the studies addressing completeness, only six (32%) specifically examined reporting completeness [16,22,28,32,34,39], with the remaining studies concentrating on the completeness of indicator data, Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Summary of main findings.

https://doi.org/10.1371/journal.pone.0316498.t002

The degree of completeness in reporting varied significantly among studies, with reported rates ranging from less than 50% to 100%. For example, district health offices in Jimma zone received 52.8% complete reports [16], while rates were 75% in the Benshangul Gumuz region [32], 86.4% in east Gojjam Zone [22], and 100% in two districts across Oromia and Gambella regions [34]. Adane et al [28] observed varying completeness levels across health services; for instance, immunization had higher completeness rates compared to other services, where rates fell below 50%.

Fourteen studies assessed data completeness using various service indicators, but the summary measures were inconsistent. Most studies lacked denominators when reporting completeness percentages, which hindered result aggregation. Only three studies, conducted in Addis Ababa [15], the Oromia region [39], and the Amhara region [31], reported completeness exceeding 90%. Four studies reported completeness rates between 80 and 90% [19,20,24,27], while the remaining studies fell below 80%. With a completeness benchmark of 90%, only three out of the 14 studies (21%) indicated satisfactory data completeness in Ethiopia.

When considering both report and document completeness collectively, only four out of 19 studies (21%) reported completeness rates exceeding 90%. Notably, completeness rates reached as low as 53.6% in the Tigray region [25]. The diversity in reporting practices, utilization of different indicators, and regional disparities highlight significant gaps in data quality, particularly concerning completeness, in Ethiopia.

Consistency of routine health data.

All studies included in the analysis provided information on data consistency, either qualitatively or quantitatively. Internal consistency was assessed in eighteen studies, which compared source documents (registers) with reported data using various indicators, detailed in the respective studies. Additionally, seven studies examined external consistency, evaluating the alignment of reported data with other independent sources or benchmarks. These assessments aimed to understand discrepancies, such as over-reporting or under-reporting, enhancing the overall reliability of data accuracy evaluations.

Internal consistency (accuracy): Internal consistency was examined in eighteen studies [15,17–25,27,28,30–35,37,39] by comparing source documents (registers) with reports using various indicators (as detailed in the respective studies). However, inconsistent reporting of denominators hindered the quantitative pooling of accuracy levels. Accuracy percentages reported ranged widely from 38.9% [25] to 90.5% [31], with one study exceeding 90% accuracy [31], and another falling between 80% and 90% [35]. Some studies noted discrepancies such as over-reporting in certain indicators and under-reporting in others, adding complexity to accuracy assessments.

The prevailing trend indicates significant over-reporting of immunizations and maternal and child health services across studies [16,19,22,24,27,28,30,34,37,39]. Child immunizations often exceeded documented figures in registers, consistently observed in all included studies. Similarly, maternal health indicators like ANC, PNC, family planning, and deliveries were frequently reported in excess [15,22,24,27,28,32,34,37,39]. Notably, one study highlighted deliveries in HMIS being reported 30.89 times more than recorded in registries [38]. While some studies reported acceptable levels of accuracy [34,37], instances of under-reporting were also noted [32]. Conversely, adverse events and disease states such as tuberculosis, pneumonia, severe acute malnutrition, and malaria were consistently under-reported [22,28,32,37,39], highlighting discrepancies in reporting accuracy across health indicators.

External consistency: Seven studies assessed the external consistency of HMIS reports by comparing them with various sources, including WHO/UN estimates of immunization [26], population-based surveys [16,19,33,36], Emergency Obstetrics and Newborn Care Assessment data [38], and JSI coverage data, as well as biomarker-linked serosurveys for immunizations [29]. The findings consistently revealed that the HMIS overestimated indicators for maternal and child health. For instance, comparisons with the 2016 and 2019 Ethiopian Demographic and Health Surveys (EDHS) revealed that HMIS reported significantly higher DPT3 coverage: 96% versus 53% in 2016 and 96% versus 61% in the 2019 mini-DHS [29]. Similar discrepancies were noted for the other vaccines and fully vaccinated children. Despite inflated vaccination reports, one in eight early neonatal deaths have been reported [36]. Although the HMIS aligned somewhat with population-based surveys, it consistently reported higher values. This trend persisted across studies, where maternal health services were more likely to be over-reported (20%) than under-reported (6%) [33]. While the HMIS tended to exaggerate maternal and child health indicators, certain outcomes, such as early neonatal deaths, were under-reported.

Consistency with related indicators: Consistency with related indicators: Several studies explored the coherence between related indicators, revealing significant disparities. For example, in certain districts, there was a 30%–63% higher attendance for first ANC visits compared to children receiving their first dose of DPT1 [16]]. Additionally, 44% of facilities had ratios greater than 1 for ANC4/ANC1 and DPT3/DPT1, indicating more cases of ANC4 and DPT3 than their respective denominators [16]. The inconsistency persisted across comparisons of syphilis testing and ANC1 [34], total births, PNC, and DPT [36], underscoring inconsistencies in related indicator comparisons.

Consistency over time: Three studies [16,34,36] investigated trends in maternal and child health service utilization over time, revealing an overall increase. These indicators remained consistent at the national level [36] or within specific districts [16,34]. However, variability was noted in neonatal and child health indicators across less than half of Ethiopia’s regions, suggesting inconsistent trends over time.

Outliers: Two studies utilized outliers to evaluate data quality. One study detected 4.03% outliers [16] while the other study identified a mix of extreme and moderate outliers across various indicators [36].

Timeliness of routine health and nutrition data.

Thirteen studies [15,16,18,20–24,27,28,31,34,39] assessed the timeliness of reports, revealing inconsistent results ranging from 41.87% [15] to 93.7% [18]. Two studies reported timeliness above 90% [18,28], while four studies fell between 80% and 90% [20,22,24,39]. However, 54% of the studies reported timeliness below 80%, with five out of these seven studies indicating timeliness below 70%.

Overall quality of routine health data.

Seven studies [15,18,21–24,39] synthesized various indicators to assess overall data quality. None of the studies reported data quality equal to or greater than 90%. Three studies reported data quality above 80% [22,24,39], while two studies reported 74% [21] and 76% [15]. The remaining two studies reported lower data quality percentages of 58% [23] and 51% [18]. These findings collectively indicate unsatisfactory overall data quality, falling short of the expected standard of 90%. The summary of findings from the included studies is shown in Table 2.

Nutrition data quality.

Five studies [19,28,35–37] evaluated data quality for nutrition indicators, revealing lower quality compared to other child and maternal health indicators. Severe acute malnutrition was notably under-reported [28,36,37], with accuracy levels ranging from 55% [37]to 97% [35]. Nutrition indicators also showed lower completeness rates, with only half of the reports submitted on time [28,36,37]. Significant gaps in external consistency were observed, particularly for deworming and vitamin A supplementation [36]. Overall, data quality for nutrition indicators in HMIS did not meet the standards seen in other indicators. Since nutrition indicators are not part of WHO’s quality assessment, limited research has evaluated their quality, suggesting potential for further data quality issues in this area. The main findings regarding nutrition data quality are shown in Table 3

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Main findings of studies on nutrition data quality.

https://doi.org/10.1371/journal.pone.0316498.t003

Challenges undermining routine health and nutrition data quality.

Supportive supervision and feedback were frequently cited factors influencing data quality in eleven studies [15,18,20,21,23,27,28,30–32,37]. Lack of training was identified in seven studies [18,21,22,26,28,30–32], while the absence of data management oversight was noted in six studies [19,21,25,27,28,31].

Various challenges influencing data quality were highlighted across the studies, including issues such as mobile phone reporting without proper documentation, inadequate registration practices, illegible data, and negligence. Competition manipulation, insufficient competency, failure to review performance, and limited experience sharing were also significant concerns. Neglecting institutional data quality assessments, lack of commitment, and the absence of essential tools, such as tally sheets, further compounded these challenges. Moreover, issues such as poor integration of data systems, the complexity of forms, and parallel reporting added to the complexity alongside language barriers in health posts and inappropriate denominators.

Additionally, factors such as limited access to computers, the Internet, electricity, fear of reporting low service coverage, and difficulties in understanding registration forms were identified. Inadequate knowledge and skills in data quality checks, lack of interest in Routine Health Information Systems due to low motivation and work overload, and false (over)reporting for recognition and promotion were also noted. Moreover, the absence of accountability for false reporting, peer-to-peer learning, incentives, commitment to digital literacy, challenges specific to urban areas, and issues in higher-tier systems contribute to the overall complexity of ensuring robust data quality in HMIS [15,18,20–23,26–28,30–33,35,37,38].

Discussion

We systematically reviewed databases to evaluate routine health and nutrition data quality across Ethiopia, focusing on completeness, consistency, and timeliness within the HMIS. Our analysis revealed pervasive shortcomings: HMIS data consistently fell short of the established standards, particularly in terms of completeness and consistency. Although timeliness showed some improvement, there were clear opportunities for improvement. Nutritional data present significant challenges, highlighting the critical need for improved accuracy. Overall, our findings emphasize the urgent need for systematic enhancements in data governance and reporting practices to bolster the reliability of healthcare information in Ethiopia.

High-quality healthcare data are indispensable for informed decision-making at all levels of the health system. It supports strategic policy formulation and efficient resource allocation and enhances decision-making by consolidating lower-level data. Conversely, poor data quality undermines the decision-making processes, leading to suboptimal outcomes [41]. Enhanced data quality empowers organizations to minimize uncertainty, boost productivity, and deliver healthcare services more effectively [3]. Furthermore, it mitigates risks to organizational reputation and lowers healthcare costs with potentially significant financial benefits [42].

The assessment of data quality within Ethiopia’s HMIS reveals multifaceted challenges and important insights into the reliability of health and nutrition indicators. Maternal and child health indicators were prominently featured across the studies, reflecting their critical role in assessing data quality. However, variability in indicator specification has been noted, with some studies omitting specific metrics, which complicates standardized assessments and global comparisons. The limited use of comprehensive WHO tracers further underscores the inconsistencies in applying standardized indicators across evaluations, highlighting the need for improved uniformity in reporting practices.

Completeness rates varied significantly across regions and health services in Ethiopia, ranging from under 50% in some districts to 100% in selected areas of Oromia and Gambella. This disparity points to an unevenness in reporting practices and challenges within specific health sectors. Notably, immunization services often demonstrated higher completeness rates than other health indicators, reflecting the varying priorities and capacities in data reporting. However, inconsistent reporting practices and the use of different metrics hinder their ability to uniformly assess and benchmark data quality against international standards.

When comparing Ethiopia’s data completeness challenges with other countries like Ghana and Tanzania, parallels emerge in disparities across regions and health sectors. Ghana faces under reporting and inconsistencies in certain districts, influenced by logistical challenges and varying capacities in data management [43]. Similarly, Tanzania contends with regional variations due to infrastructure limitations and inadequate healthcare personnel training [44]. In contrast, Kenya, Nigeria and Bangladesh have improved completeness through technology and training, emphasizing standardized reporting and enhancing data reliability for informed decision-making and improved health outcomes [45–47].

Internal consistency assessments, conducted in eighteen studies, compared reported data with source documents (registers), revealing a wide range of accuracy levels. These assessments highlighted instances of over-reporting in immunizations and maternal health services, where reported figures exceeded those recorded in the registries. This discrepancy suggests potential inaccuracies in data entry or reporting biases that could skew the overall health assessments and resource allocation decisions.

External consistency evaluations, explored in seven studies, scrutinized the alignment of HMIS data with independent sources, such as WHO/UN estimates and population-based surveys. The findings consistently indicated over estimations in maternal and child health indicators compared with benchmarks that are more reliable. Disparities were particularly evident in vaccination coverage, where HMIS reports often exceeded demographic and health survey findings. Such inconsistencies underscore the challenges of ensuring data reliability across different health sectors and highlight the importance of robust validation mechanisms to enhance data accuracy.

Ethiopia’s HMIS data, as assessed through external consistency evaluations, shows notable discrepancies compared to other countries. Similar evaluations in countries like Kenya and Nigeria have also revealed challenges with over estimations in maternal and child health indicators within their respective HMIS. However, while these countries have implemented strategies to improve data accuracy, such as enhanced validation processes and standardized reporting guidelines [47], Ethiopia faces ongoing challenges in aligning HMIS data with more reliable benchmarks like WHO/UN estimates and demographic surveys. These disparities underscore the broader global challenge of ensuring accurate health data reporting across diverse health systems and highlight the critical need for strengthened validation mechanisms and improved data governance practices to enhance data reliability and inform evidence-based decision-making in healthcare.

There is a need to enhance the completeness of data in Ethiopia’s routine HMIS. Although a few studies have achieved completeness rates of >  90%, most have not met this standard, highlighting the need for improvement. Standardizing reporting practices and ensuring the consistent use of summary measures could help address regional variations and enable more accurate assessments of the quality of health data nationwide. These efforts are essential for improving the reliability and usefulness of health information for evidence-based decision-making and policy development.

Compared with other countries, countries like Uganda have made strides in achieving high completeness rates by implementing standardized reporting practices and rigorous data validation mechanisms [48]. In contrast, Ethiopia faces greater disparities across its regions and health services, where achieving uniform completeness remains a challenge due to inconsistent reporting practices and varying capacities in data management. While all these countries prioritize enhancing data quality for evidence-based decision-making, Ethiopia can benefit from adopting similar strategies to improve the reliability and utility of its health information nationwide.

Outlier analysis in two studies provided additional insights into data quality issues, identifying extreme and moderate deviations in the reported indicators. This methodological approach helps identify anomalies that could distort the overall assessment of health data reliability and accuracy, thereby informing targeted interventions to improve data management practices.

The timeliness of data reporting, as assessed in 13 studies, varies widely across different health facilities and regions. While some reported timeliness rates above 90%, others fell below acceptable thresholds, indicating systemic challenges in ensuring timely data submission. Addressing these challenges is critical for maintaining up-to-date health information that is crucial for effective planning and response to public health needs.

Countries like Ghana and Nigeria also employ outlier analysis to pinpoint discrepancies that affect data reliability, supporting targeted interventions to enhance reporting accuracy [49]. However, Ethiopia’s timeliness of data reporting, as evidenced by 13 studies, shows significant variability compared to countries like Rwanda and Uganda [50,51], where efforts to streamline reporting processes have resulted in consistently higher timeliness rates across health facilities. Addressing these disparities through improved infrastructure and standardized reporting protocols is essential for Ethiopia to maintain timely and accurate health information, crucial for effective public health planning and response.

The results of seven studies assessing data quality in Ethiopia underscore persistent shortcomings, with none meeting the desired 90% standard and many falling below 80%. These deficiencies across accuracy, completeness, and timeliness highlight urgent needs for systematic improvements in the country’s health information systems to enhance reliability and support informed decision-making and resource allocation.

Compared with countries like Kenya and Rwanda, Ethiopia faces greater challenges in data quality [51,52]. These nations have implemented robust data governance and reporting strategies, resulting in higher overall data quality scores and consistent adherence to international standards. These comparisons emphasize the imperative for Ethiopia to adopt similar systematic improvements in its health information systems to ensure reliable health data for effective public health management.

This study has several strengths. It adopts a comprehensive approach by reviewing multiple studies and providing a broad and detailed analysis of data quality within Ethiopia’s HMIS. It assesses both internal and external consistency, outlier analyses, and timeliness of data reporting, thus offering diverse perspectives on data reliability. This multifaceted approach enhances the robustness and depth of the findings, allowing for a nuanced understanding of the complexities and challenges within the HMIS. Additionally, by synthesizing findings across various health indicators and regions, this study contributes to a more holistic view of the strengths and weaknesses of Ethiopia’s health information systems.

However, this study is not free of limitations, owing to the variability in methodologies among the reviewed literature, which may hinder the direct comparability of findings. Each study included in the review may have employed different criteria, metrics, or analytical techniques, potentially skewing the synthesis of the results. Moreover, reliance on published studies introduces publication bias, as unpublished data and negative results may not be represented, potentially influencing the overall assessment of data quality in Ethiopia’s HMIS. Furthermore, the geographical focus on specific regions and health indicators within Ethiopia limits the generalizability of our findings to the entire country. Variations in health service delivery and data management practices across different regions may not be fully captured, thus restricting a comprehensive understanding of nationwide data quality challenges. Given the recent instability of the country and internal displacement of populations, coverage data might be influenced. Some areas, which have incoming population, may have above 100% coverage, this scenario might affect recent studies and external consistency results might be influenced.

In conclusion, the synthesis of findings underscores significant challenges in data quality within Ethiopia’s HMIS, marked by inconsistencies in reporting practices, regional disparities, and limitations in standardization across health indicators. Despite these challenges, this study’s comprehensive review highlights opportunities for improvement, including enhanced training, standardized reporting protocols, and strengthened data management oversight. Addressing these issues is crucial for enhancing the reliability and utility of health data, and ultimately facilitating more effective policy formulation, resource allocation, and health interventions tailored to the diverse needs of Ethiopia’s population.

Supporting information

Acknowledgments

None.