Study design and period

This qualitative study was conducted over a specified period in five strategically selected regions of Ethiopia, including the capital city administration of Addis Ababa and four other regions. The study employed an exploratory qualitative approach, employing both key informant in-depth interviews (KII) and focus group discussions (FGD) to explore the intricacies of DHIS2 utilization in the context of primary healthcare (PHC). Data collection and analysis were conducted over six months, from October to March 2024, ensuring thorough coverage and depth in the findings.

The KII respondents included health managers of primary healthcare institutions such as district hospitals, health centres, and health posts. Additionally, health service leaders/administrators such as heads of district health offices, zonal health departments, regional health bureaus (RHBs), and the federal ministry of health (MOH) were also included in the interviews. The FGDs were conducted with performance management team leaders of district health facilities and health managers from selected districts.

Study area

The study was primarily conducted at primary health care (PHC) institutions in Ethiopia. These institutions include district hospitals, health centres, health posts, and district health offices, as defined by the Ministry of Health (MOH). The selected PHCUs for the study were chosen from regions representing four different agroecology and economic areas/zones of the country. Specifically, two agrarian regions (Amhara and Oromia), a cash crop producing region (Sidama), a pastoralist region (Somali), and an urban area were included.

Qualitative data was collected through interviews with health managers at zonal health departments, RHBs, and the MOH. Furthermore, interviews or discussions were conducted with performance monitoring team (PMT) members and HIS focal persons at their respective health facilities.

Sample size and sampling technique

The determination of sample size was guided by the principle of information saturation, ensuring that sufficient data were collected to achieve thematic saturation across diverse sub-groups. A combination of key informant interviews and focus group discussions enabled the exploration of multiple perspectives and experiences related to DHIS2 utilization.

Purposive sampling techniques were employed to select participants based on their roles within primary healthcare institutions, level of training, and professional experience. This method ensured the inclusion of a diverse array of stakeholders, from health managers at district hospitals to performance monitoring team leaders at health facilities. In total, we conducted 30 KIIs and 16 FGDs. Most focus groups consisted of 8 participants, although a few included between 7 and 9 participants, resulting in approximately 132 participants across the 16 FGDs.

Sampling procedure

Convenience and purposive sampling strategies were utilized to identify and recruit participants for focus group discussions and key informant interviews, respectively. Participants were selected from primary healthcare units (PHUs) and health administration offices spanning federal to district levels, ensuring representation from various tiers of the healthcare system. study participants were distributed across the four major livelihood systems (agrarian, cash-crop, pastoralist and urban) to produce a robust and geographically triangulated result.

Data collection tools and procedures

Structured and unstructured interview guides were developed in English and subsequently translated into local languages, including Amharic, Afan Oromo, and Somali, to facilitate effective communication with participants. A team of trained data collectors, comprising individuals with backgrounds in health, information technology, and behavioural sciences, were deployed to conduct interviews and focus group discussions. Prior to data collection, all team members underwent intensive training to ensure proficiency in administering questionnaires, capturing responses, and maintaining ethical standards.

Outcomes and measurements

The qualitative study aimed to explore key dimensions of DHIS2 implementation, including data availability, accessibility, accountability, and utilization within the PHC context. These outcomes were assessed through in-depth interviews and focus group discussions, allowing for a nuanced understanding of the multifaceted impact of DHIS2 on healthcare delivery and decision-making processes.

Data analysis

The data analysis process seamlessly commenced alongside data collection in the field, with verbatim transcripts promptly translated into English for meticulous examination. Employing thematic analysis, a rich tapestry of recurring patterns, themes, and insights emerged from the qualitative dataset. ATLAS.ti software emerged as a crucial ally, facilitating the organization, coding, and interpretation of qualitative data, ensuring both methodological rigor and transparency throughout the analytical journey.

Within the thematic analysis, a nuanced exploration unveiled several pivotal themes and their corresponding subthemes. Anchored under the overarching theme of the perceived impact of DHIS2 on healthcare services’ data quality, the timeliness of routine HMIS data took center stage, highlighting the system’s notable influence.

Another substantial sub-theme that emerged was the "Effect on Data Completeness," capturing community members’ perceptions of the comprehensiveness and inclusivity of health data captured through DHIS2. Subthemes within this realm ranged from the degree of pertinent health indicators captured to the integration of marginalized populations in health data reporting.

Moreover, the sub-theme of "Effect on Data Accuracy" delved into community members’ perceptions of the reliability and precision of health data collected through DHIS2, shedding light on the system’s impact on data integrity.

Additionally, participants underscored the significance of the "Effect on Reduction of Discrepancies Between Reports," highlighting DHIS2’s role in harmonizing data across various health facilities, resolving disparities, and fostering consistency in health data reporting.

Under the sub-thematic lens of "Content (Indicator) Completeness," community members emphasized the imperative of capturing a comprehensive array of health indicators within DHIS2 to drive evidence-based decision-making and program planning.

Furthermore, other notable themes emerged, including the Perceived Impact of DHIS2 on Data Availability and Accessibility, its effect on Data PHCU Accountability, the influence of DHIS2 on Equity of data quality and health services, as well as the serendipitous Unexpected Benefits unearthed through its implementation.

Ethical considerations

Ethical clearance was secured from IRBs of Public Health Institutes in all five regions and Addis Ababa city administration. Permissions and administrative support were obtained from the federal Ministry of Health and regional health bureaus. All participants provided verbal informed consent, emphasizing voluntary participation, confidentiality, and the right to withdraw without repercussions. Formal documentation was promptly generated in the form of confirmation emails and letters from each IRB. Stringent ethical standards were followed to protect participants’ rights and well-being throughout the study.

Perceived impact of DHIS2 on data quality at PHUs in Ethiopia

Overall, participants in the qualitative study unanimously agree that DHIS2 has significantly improved the quality of data. The study included participants from all regions and employed methods such as Key Informant Interviews and Focus Group Discussions. Although there may be some differences in certain areas, there is a consensus across regions about the positive impact of DHIS2 on key dimensions of data quality, such as timeliness, availability, accessibility, and accuracy of health and nutrition data. The following paragraphs provide a detailed discussion of each of these components, explaining the improvements observed in timeliness, availability, accessibility, and accuracy, all thanks to DHIS2.

Effect of DHIS2 on timeliness of routine HMIS

The finding indicates a strong consensus among respondents from all five regions regarding the overall improvement in data quality dimensions compared to pre-DHIS2 periods. Specifically, there is notable agreement on the substantial enhancement in timeliness of data. A PHCU head from one region emphasized the stark improvement in timeliness following DHIS2 implementation, highlighting a significant shift from previous limitations in monitoring data timeliness to clear enhancements in data management reports, saying:

"Before DHIS2 came into play in our district, we struggled with a glaring gap in keeping tabs on data timeliness. But once we implemented it, the transformation was palpable–our data management reports now boast a level of timeliness that was previously unimaginable."

           PHCU head in one of the regions

The consensus among many PHCU leaders, ICT technicians, and other participants is that the implementation of DHIS2 has significantly improved data management processes. This improvement is attributed to the introduction of specific timeframes for data entry and completion, ensuring the timely and accurate input of data. Moreover, DHIS2 standardizes data collection formats, which enhances consistency and overall quality throughout the system.

"The introduction of DHIS2 not only streamlines data entry and completion with its predefined timeframes but also assures the validation, timeliness, and accuracy of the entered data. Embracing real-time data entry further enhances data quality. Moreover, DHIS2’s uniform data collection formats guarantee consistency and comparability across the entire system, fostering reliable analysis and decision-making."

           ICT head from an urban PHCU facility

However, in one of the regions, there was a prevailing sentiment that although there has been improvement in data completeness, the timelines within DHIS2 have not changed. This is primarily due to the strict deadline for submitting reports, which is currently set for the 22nd of every month.

"In our country, health facilities must submit reports by day 21, with Health Posts sending data to health centers by day 23. Health centers finalize reports by day 26. Notably, some regions submit reports as early as day five, highlighting improved efficiency."

           Head of a PHCU in one of the districts

In alternative regions, participants expressed apprehension regarding variations in reporting intervals, disparities among individual reports, and discrepancies encountered when soliciting identical reports for the same timeframe. Although these concerns persist to some extent, there has been discernible progress. One FGD participant remarked:

"As previously mentioned, disparities in reporting times and inconsistencies among reports from different individuals were observed. Even when requesting identical reports from the same timeframe, disparities persisted. It is noteworthy that while these challenges have not been entirely eradicated, there has been a degree of improvement."

Effect of DHIS2 on data completeness among the PHUs

Participants in the study have emphasized the beneficial influence of DHIS2 in improving data completeness within PHCUs and along the reporting hierarchy. They substantiate their claims by pointing out that reports generated through DHIS2 are thorough and inclusive, crediting this to the system’s inherent mechanisms for ensuring completeness.

"In our previous reliance on Excel sheets within the former HMIS, error detection in reporting was conducted manually. However, with DHIS2, if figures are not inputted into the ’by-marginal age’ classification, the system triggers a red alert, prompting data rectification."

           Head of a PHCU in Addis Ababa

This notion was further supported by a Health Information Technician (HIT) from one of the PHCUs.

"Before implementing DHIS2 software for reporting and managing our routine health facility data, our reporting lacked completeness, making it difficult to assess the quality of information. Moreover, data indicators and elements were typically aggregated and transmitted solely as figures. Now, we have access to both indicators and data elements."

On the contrary, regarding potential errors in routine health facility reports, they attribute them to limitations or mistakes in recording and record-keeping practices, rather than shortcomings in the DHIS2 software.

"Often, when we conduct quality checks, the errors we encounter originate from the registers. Therefore, I cannot attribute them solely to issues with Health Information Technicians (HITs)…"

           Head of a district PHC health office from one of the regions.

They also emphasize the significance of establishing and reviewing routine health facility data by the Program Management Team (PMT) before forwarding it to the next higher level.

"Most of the time, you’ll find accurate data on DHIS2 because errors are corrected during PMT and case team meetings," noted the Head of a PHCU from one of the regions.

Some PHCU leaders and HIT professionals also argue that DHIS2 achieves completeness not only by reporting expected data but also by enabling the inclusion of a wide range of comprehensive indicators.

"I firmly believe that DHIS2 has revolutionized our programs. In the past, the ANC program was overlooked in DHIS2, but now it is seamlessly integrated, allowing us to effortlessly track the number of pregnant mothers receiving services and ANC visits. This inclusion has catalysed a positive shift in reducing maternal mortality rates."

           FGD discussant from one of the regions

Effect on data accuracy

Study participants unanimously concurred that the implementation of DHIS2 for routine data management and reporting has led to a significant improvement in data accuracy, facilitating seamless decision-making processes.

"DHIS2 advances beyond traditional HMIS methods, which rely on manual processes. Its streamlined approach enhances accuracy and speeds up data entry and management, making it a more efficient and preferred choice for many healthcare organizations."

           Head of a PHCU from one of the regions

While some challenges persist, such as sporadic inaccuracies in reports, DHIS2 provides convenient access to precise data for prompt decision-making.

“During data entry, multiple cross-checks are conducted. Initially, accuracy is verified, followed by ensuring completeness and adherence to required timelines. Additionally, the system conducts its own checks, including validation of components.”

           HIT from one of the regions

They also indicated that DHIS2 has enabled corrections and enhancements to inaccurate data, often addressed during PMT and case team meetings, thereby bolstering the credibility of data within the DHIS2 system.

“…In my assessment, while there are areas needing improvement to enhance decision-making, our progress thus far is commendable given our early stages. A notable incident involved ART data, crucial for funding medication. A 50% discrepancy between our report and the actual client numbers prompted urgent data cleaning. Addressing such discrepancies promptly is essential to ensure all clients receive their medication as needed."

           HIT from one of the regions

“… Precise data is often encountered within DHIS2, as discrepancies are typically resolved during Program Management Team (PMT) and case team gatherings.

           PMT member FGD discussant from one of the regions

Effect on reduction of discrepancies between reports

Across various regions, participants perceive DHIS2 as a key factor in notably mitigating data discrepancies in reports. They view DHIS2 as a robust instrument for gathering, validating, and analysing health data in Ethiopia, thereby fostering harmonization, and streamlining within the process. Consequently, this culminates in enhanced data quality, facilitating informed decision-making and policy formulation.

"… when contrasting the manual process with the current DHIS2 system, noticeable disparities emerge. Previously, reports often exhibited temporal inconsistencies, where data from the same period varied between iterations. Furthermore, discrepancies in recording and reporting were commonplace."

           A regional HIT expert

Content (indicator) completeness

Study participants assert that DHIS2, incorporating an array of new tools, indicators, and service categories, empowers users to generate diverse datasets and outcomes. This expanded functionality significantly enhances the capacity for comprehensive health monitoring and analysis.

"In its initial stages, the HIMS lacked several crucial program and service indicators. With the adoption of DHIS2, however, there has been a gradual expansion in the indicator repertoire. Starting at 108 indicators, the system progressively grew to 122, then 131, and now encompasses approximately 171 indicators. This extensive range now comprehensively covers various program metrics, reflecting the system’s evolution and adaptability"

           HIT expert from a region

Likewise, a district head from one of the Primary Health Units (PHUs) underscored the significance of DHIS2 in mitigating data disparities across reports, stating:

"The system now includes 188 indicators, up from 131, enabling a comprehensive evaluation of health status. With DHIS2, our service portfolio has significantly expanded, providing tools that can effectively generate valuable data and outcomes."

           District PHCU head from a region

Perceived impact of DHIS2 On data availability and accessibility

Ever since its inception, DHIS2 has wielded a profound influence on the accessibility of data in numerous low- and middle-income countries (LMICs), heralding a new era of centralized data management and real-time access in the realm of public health. Through its intuitive interfaces, customizable reporting tools, and facilitation of collaborative efforts among stakeholders, DHIS2 stands as a beacon of progress, significantly bolstering health outcomes and fortifying healthcare systems across LMICs.

In Ethiopia, the consensus among nearly all participants in this study resonates with the transformative impact of DHIS2 in breaking down barriers to data access, transcending geographical and temporal constraints. For instance, a PHCU director from one of the regions grappling with data accessibility issues eloquently articulated the pivotal role of DHIS2, stating:

"DHIS2 revolutionizes data access, liberating us from HMIS constraints. Previously, accessing data was restricted by time and place. Now, with DHIS2, data is at our fingertips anytime, anywhere, contingent upon stable connectivity. It’s a transformative leap in accessibility and efficiency."

           Head of a PHCU in one the regions

Another key informant from the region echoed this sentiment, affirming,

"The web-based nature of DHIS2 offers a distinct advantage: accessibility from any location with just a username and password. Previously, with paper-based reporting, accessing data beyond the facility was cumbersome. DHIS2 eliminates these barriers, providing on-demand data access, anytime, anywhere."

Study participants emphasized the importance of accessing HIMI data via DHIS2 even on weekdays when specific staff members, like HITs, might be unavailable but the data remains crucial for decision-making. A seasoned HIT from Addis Ababa elaborated on this need for continuous access to data for informed decision-making.

"On Saturdays and Sundays, we are off duty, but we’re still committed to assisting managers remotely if needed. Providing essential information during these days is our priority. Additionally, with an official letter, anyone can access the data, ensuring equitable availability."

           Experienced HIT from Addis Ababa

Additional participants fervently highlighted DHIS2’s role in ensuring offline data accessibility. They marvelled at its capacity to facilitate seamless data collection, entry, analysis, and decision-making in areas with sporadic internet connectivity. DHIS2’s potential to sustain operations despite limited or unreliable online access left a profound impression on them.

"In our region, numerous health facilities lack internet, computers, or reliable power for DHIS2 operation. Offline data access has emerged as a vital solution, enabling facilities with computer and electricity access to input data offline. Subsequently, data is transferred via flash drive or external disk to an internet-enabled location, ensuring accessible data retrieval whenever necessary."

           Regional expert working on HMIS and data revolution

Furthermore, participants underscored DHIS2’s profound impact on health data storage and accessibility. Its centralized database, robust backup and recovery systems, user access control, role-based access, and web-based interface ensure the secure, confidential, and user-friendly storage and retrieval of health data. DHIS2’s array of features guarantees long-term data accessibility and usability within the healthcare system, facilitating multi-person access and promoting data security and confidentiality. DHIS2 served as a centralized database for storing health data, ensuring that all information is stored in one secure location. This centralized approach facilitated long-term storage by preventing data fragmentation and loss.

"As a HIT deeply engaged with health data management, I’ve witnessed first-hand the transformative power of DHIS2’s centralized database. It acts as a secure repository, consolidating all health data into one centralized hub. This approach not only safeguards against fragmentation and loss but also ensures long-term storage, enhancing data integrity and accessibility."

           Regional HIT technician

DHIS2 incorporates robust data backup and recovery mechanisms to safeguard against data loss. Regular backups were performed to ensure that data is protected and can be restored in the event of system failures or disasters.

DHIS2 allows administrators to set user access permissions, controlling who can view, edit, or delete specific data. This ensured that only authorized personnel have access to sensitive health information, promoting data security and confidentiality.

DHIS2 supported role-based access control, allowing different users to have access to specific data based on their roles and responsibilities within the health system. This ensured that relevant stakeholders, including healthcare providers, managers, and policymakers, accessed the data they need to perform their duties effectively.

"As head of a Primary Healthcare Unit, DHIS2’s role-based access control has been instrumental. Its tailors’ data access to users’ roles, ensuring our healthcare providers, managers, and policymakers access the precise information required for their responsibilities. This targeted access enhances efficiency and empowers effective decision-making within our health system."

           Head of a Primary Healthcare Unit from a city

DHIS2 is accessible through a web-based interface, allowing authorized users to access health data from any location with internet connectivity. This promotes multi-person accessibility by enabling healthcare teams and stakeholders to access data remotely, facilitating collaboration and decision-making.

Participants in the study have articulated DHIS2’s provision of expansive and user-friendly data management capabilities. Authorized individuals, spanning staff members, managers, and stakeholders, can seamlessly access, generate, and utilize healthcare data. DHIS2’s open-source architecture guarantees widespread accessibility, while robust permissions and password controls safeguard data integrity. Moreover, DHIS2 facilitates programmatic access to healthcare data, streamlining data management and analysis within the healthcare ecosystem.

"With permission, anyone can access data through DHIS2, an open-source platform that allows universal usage. DHIS2’s being an open-source platform, data accessibility transcends boundaries. Permission grants universal usage, empowering stakeholders to harness data responsibly. DHIS2 offers unparalleled data availability, surpassing HMIS. Authorized individuals can access it from anywhere, facilitating informed decision-making. Data analysis usernames further empower staff and managers to glean insights while ensuring data integrity."

           District PHCU manager from one of the regions

Perceived impact of DHIS2 On Data PHCU accountability

DHIS2 is credited with bolstering accountability on various fronts, including error detection and correction, transparency enhancement, and fostering ownership and responsibility. Study participants highlighted DHIS2’s role in accountability, noting its adeptness in identifying and rectifying mistakes swiftly through robust validation features. Furthermore, DHIS2’s audit trail capability ensures transparent tracking of data activities, instilling confidence in data integrity and minimizing errors. These accountability measures not only enhance data quality but also cultivate trust in health information systems.

“…For instance, when editing data in DHIS2, the platform records the time and location of the edit, enhancing accountability by providing clear visibility into data modifications."

           HIT from a PHCU

DHIS2 is lauded for its role in enhancing accountability through improved transparency, achieved through various mechanisms. Central to this is its provision of a centralized data platform, facilitating seamless oversight and management. Authorized users benefit from unified access to data, promoting transparency as stakeholders monitor health indicators and outcomes in real-time. DHIS2’s additional features, including data validation rules and audit trails, further bolster transparency by identifying errors and unauthorized alterations, ensuring data precision and integrity. Ultimately, this fosters informed decision-making, as decision-makers rely on reliable, transparent information for evidence-based planning.

“… We strictly adhere to protocol; information is only released upon presentation of an official letter from the health bureau or sub-cities. It’s important to note that the data I possess may vary from incoming information."

           PMT member FGD discussant from Addis Ababa

Furthermore, DHIS2 is noted for its facilitation of data sharing and reporting across different tiers of the healthcare system, spanning from local facilities to national entities and international partners. This transparency fosters collaboration among stakeholders at all levels, facilitating the exchange of information and fostering mutual accountability in achieving health objectives. DHIS2’s accountability mechanisms include enhancing data management transparency, ensuring data accuracy and integrity, and facilitating collaboration and information exchange throughout the healthcare continuum.

DHIS2 empowers users within the healthcare sector by fostering ownership and responsibility in data management. Through decentralized data entry and management, local health facilities and personnel take charge of their data, promoting accountability and informed decision-making at the facility level. Role-based access control ensures transparency by assigning specific responsibilities to users. Administrators designate roles like data entry clerks and program coordinators, ensuring everyone understands and fulfils their duties. This approach not only enhances data integrity but also cultivates a culture of accountability and transparency within the system. DHIS2 thus goes beyond being a technological platform; it becomes a catalyst for positive change in healthcare delivery.

Furthermore, DHIS2 fosters collaboration and teamwork by enabling seamless data sharing and communication among stakeholders. Equipped with data visualization tools and reporting dashboards, DHIS2 facilitates collective efforts in data analysis, interpretation, and action planning. This collaborative approach nurtures a shared sense of ownership over health data and its outcomes. In essence, DHIS2 empowers users to actively participate in data management, delineating clear roles and responsibilities while promoting collaboration and communication among stakeholders. Consequently, this cultivates a culture of accountability and ownership within the health system, culminating in elevated data quality and enhanced health outcomes.

Effect of DHIS2 on Equity of data quality and health services

Ensuring equity in the data quality of Maternal and Child Health (MCH) indicators in Ethiopia is paramount to accurately capturing the health status and needs of every segment of the population, especially mothers and children. This equity encompasses various dimensions within MCH indicators, including coverage and representation, accuracy and completeness, and timeliness and accessibility. DHIS2 emerges as a pivotal tool in promoting equity not only in health services but also in data quality across Ethiopia. By facilitating comprehensive data collection, analysis, and dissemination, DHIS2 has the potential to bridge gaps in information accessibility and accuracy, thereby contributing significantly to improved health outcomes for all demographic segments.

Unexpected benefits of DHIS2 implementation in Ethiopia

Participants in the study highlighted several unexpected benefits that emerged from the adoption of DHIS2 in various regions of Ethiopia. These advantages encompass a wide range of benefits, including but not limited to, the availability of reliable data storage solutions, opportunities to improve staff digital literacy, designated spaces for Health Management Information System (HMIS) operations, the creation of innovative structures for HMIS management, and the generation of employment opportunities for graduates and young individuals. Moreover, the introduction of distinct registries as primary source documents and increased involvement in data management processes were additional positive outcomes resulting from the implementation of DHIS2.

"It is more fitting for program users to provide insights. Our role is to ensure data accessibility, currency, and accuracy, empowering them to offer clearer explanations. Take the ART program, for instance; readily available data on current ART recipients, new initiates, and those not on ART enables more effective program management. Additionally, higher-level stakeholders can review and assess the program with greater depth and clarity. “

           Health Information Technician

Participants in the study identified several unexpected benefits from adopting DHIS2, including the ability to store long-term data for retrospective analysis and improved digital literacy among staff. Dedicated HMIS operation rooms streamlined workflows, enhancing efficiency and accountability. Additionally, the system created job opportunities for graduates, promoting local employment and economic empowerment while improving data quality and overall system functionality.

The discovery of unexpected benefits highlights the dynamic nature of DHIS2 implementation and its ability to produce positive outcomes beyond its original purpose. These unforeseen advantages demonstrate the system’s adaptability and its transformative impact on healthcare information management practices. By embracing emerging opportunities and addressing evolving requirements, DHIS2 showcases its potential to encourage innovation, strengthen capacity, and drive socio-economic progress in the field of healthcare.

Recognizing these unplanned benefits emphasizes the importance of continuous evaluation and stakeholder engagement in maximizing the value and long-term viability of health information systems like DHIS2. As organizations utilize DHIS2 to enhance health outcomes, acknowledging and capitalizing on these unexpected advantages will play a crucial role in optimizing the system’s effectiveness and magnifying its potential impact.