a. Study sites.

The intervention districts will be selected purposively based on the fact that the data use interventions will be implemented. The control districts will be selected from two regions to match the intervention districts based on geographical location (urban vs. rural). The health facilities from the selected districts were selected based on the following criteria: a primary public health facility, providing services for more than a year. Hereunder the characteristics of the selected districts are described.

Kinondoni and Temeke are urban districts (Municipality) in the Dar es Salaam Region, the business capital of Tanzania Mainland. The census of 2012 showed that the population of Kinondoni was 1,775,049 whereas that of Temeke District is 1,368,881.

Chamwino and Dodoma City councils are districts in Dodoma Region, the administrative capital of the united republic of Tanzania. The census of 2012 showed that the population of Chamwino was 330,543, whereas that of Dodoma city council was 410,956.

Tanga city council and Korogwe are districts in the Tanga region, one of the important cities in the coastal zone of the country. The census of 2012 showed that the population of Korogwe was 56,282 whereas that of Tanga city council was 273,332.

Mwanza city council and Misungwi District council are districts in Mwanza region, one of the important cities in the Lake zone region of the country. The census of 2012 showed that the population of Mwanza city council was 706,453 whereas that of Misungwi was 351,607.

The eight councils represents a mixture of metropolitan (Temeke & Kinondoni), urban (Dodoma City council, Mwanza city council, Tanga city council) and rural district councils (Chamwino, Korogwe and Misungwi). The study areas are suitable of generating outcomes that could easily be transferable to other similar settings in the country.

b. Description of the data use initiatives.

This implementation study is conducted in the context of an intervention that is set to promote data use for decision making and planning to improve the quality of care and health status (outcomes) of the population.

The President’s Office ‐ Regional Administration and Local Government (PO-RALG) requested assistance from UNICEF and USAID to promote and institutionalize a culture of data use at subnational level in the stride forwards to achieve universal health coverage (UHC) and strengthen PHC systems in line with the decentralization policy and related approaches such as the direct health facility financing (DHFF) . USAID engaged the Data for Implementation (Data.FI) project, to support the implementation of the request with the aim of using quality data to support evidence-based health programming and accountability.

In April 2021, UNICEF and Data.FI drafted a proposed joint implementation plan to support implementation of data use for this quality improvement initiative at different levels of the health care system, with Data.FI, providing support at national, regional and council level whereas UNICEF supports the health facility level. The intervention strategies aim to address 1) organizational determinants of data use (governance, equipment & tools and planning ) 2) technical determinants of data use (physical infrastructure, ICT infrastructure , HIS design and operability 3) behavioral determinants of data use (capability, opportunity and Motivation )

1) Develop a theory of change (TOC) for data use initiatives and document the process under which such a TOC is developed.

In order to understand the mechanism through which outcomes of data use initiatives are produced, we follow the implementation science steps throughout the conduct of this study. The steps are 1) developing the theory of change (TOC by reviewing internal program documents, attendance to various meetings on the data use initiatives and consultations with program developers and implementation teams and literature review on similar programs targeting primary health care settings 2) Testing the TOC, by developing tools and collecting empirical data in the field 3) Refining the TOC through a qualitative work consisting of in-depth interviews with key stakeholders involved in, or who have a stake in data use initiative, and focus groups and in-depth interviews with program beneficiaries, i.e., the health care mangers and frontline health care workers. The initial TOC has been developed (see Fig 1 & S1 File for a details) in order also to guide the development of data collection tools at all stages of the study.

2) Understand the initial (baseline) situation of data use in the study settings in relation to decision making and comprehensive (council or health facility) planning.

To understand the baseline situation, the four councils implementing the data use initiative will be matched equally with comparable councils where the data use interventions will not be implemented. Stratified sampling (to distinguish the level of facilities) will be used in these four councils to select a representative sample of facilities to be observed for one year after the project is implemented. This step combines quantitative and qualitative methodologies to evaluate the current state of data use in the recipient health facilities.

3) Understand the implementation processes (acceptability, adoption, fidelity and sustainability) of the data use initiatives amid the influence of contextual factors.

Complex interventions are often criticized as being a black box because it is difficult to know exactly why an intervention did (or did not) work. To address this gap and open the black box, a process evaluation will be completed using a convergent parallel mixed methods design to assess acceptability, adoption, fidelity and maintenance of the intervention processes. The convergent parallel mixed methods component refers to the integration of both quantitative and qualitative data collection methods to complete the process evaluation at the same time. The process indicators reflect the immediate outputs of the data use initiatives. In line with existing evidence on intervention uptake and sustainability, the process evaluation will elucidate how the initiatives have been implemented in the intervention councils and why [21, 27–31]. In turn, this information will be used to explain the probable heterogeneity of effects observed when measuring the effect of the program. Triangulation of data from multiple sources will be done to generate comprehensive information from this study and generate very informative body of knowledge to inform data use related policies and guidelines in the primary health care settings. All the above activities will be guided by the initial TOC developed in this study.

Data will be collected from health managers and frontline workers delivering services in health facilities. Quantitative data collection will be done using structured questionnaire to health care managers and frontline health workers who produce data at multiple points and a document review checklist in order to assesses both the implementation outcomes (acceptability and fidelity) and service outcomes (efficiency, safety and patient centeredness).

The specific research questions in this component will be as follows:

1. To what extent the data use initiative has been adopted, accepted and maintained?
2. How many users of primary health facility have been reached by the initiative?
3. To what extent do the implementers and beneficiaries of the initiative adhere to the initiative protocol?
4. Is data use initiative faced with implementation challenges? What are those challenges and how can they be addressed?

To answer the above research questions, the following objectives will have to be addressed

1. To assess the extent of adoption of the data use initiatives
2. To determine number or proportion of beneficiaries reached by the data use initiative
3. To assess the fidelity to data use initiative protocol and SOPs
4. To assess acceptability of the data use initiative by primary beneficiaries and other stakeholders
5. To assess sustainability of the data use initiative in the intervened primary health facilities
6. To determine the implementation challenges and success of the data use initiatives
7. Understand the effects of the data use initiative on health system and patient outcomes

To understand the effects of the data use initiative, we will collect data at baseline and end line in order to allow two points measures of the outcome data. In this step, to assess compliance to national and international quality standards, the study team will rely both on the analysis of the routine health information system (at the facility level and district level) and information collected from respondents. Specific research questions include, i) has the quality of primary health facility data improved following implementation of the initiative? ii) What factors have contributed to the observed changes?

Specific objectives:-

1. Assess the effect/impact of data use initiative on beneficiaries’ capabilities (knowledge, and skills to analyze and interpret data)
2. To evaluate the impact/effect of data use initiative on the quality of data collected in the primary health facilities
3. To determine the impact/effect of the initiative on the use of data for decision making and planning
4. To evaluate the effect of data use initiative on the performance of primary health facilities in terms of quality of delivered health services (availability drugs, medical supplies and equipment) and client’s satisfaction with the services.

Study component 1: Measuring effects of the data use initiative. This component aims at establishing the effects of data use initiatives on quality of care and selected health outcomes. To establish and control the counterfactual, data will be collected in the intervention and control councils. This component comprises collection of quantitative and qualitative data at two points (baseline and end line) covering a period of one year.

Sampling and sample sizes. This component will collect information from health facility documents , health care workers, and patients. The samples and sample sizes for this quantitative component are as follows

1. We will collect health facility level data to assess the overall situation of quality of care from a total of 80 health facilities (40 from intervention health facilities & 40 from control health facilities).
2. From each of the selected health facilities we will collect quantitative data from at least five participants (the chairperson of the health facility governing committee, in charge of the health facility, data focal person , in charge of RMNCAH services and in charge of outpatient services) making a total of 400 participants. This sample is adequate to conduct robust inferential analyses related to the effects of the data use initiative. Moreover, taking p = 50% and applying Cochrane formula while also accounting for refusals and the fact that some health facilities (especially the dispensaries) may not all the five staff, an addition of 10% to the minimum sample will be added to reach 440 participants.
3. Also, from the selected facilities exiting patients will be systematically selected after gender stratification following medical consultations. The exit interview with Client/patients will be approached after they have received the services and are ready to go home. Respondents eligible for interview include all exiting patients or relatives of patients (aged above 18 years). They will be sampled to ensure equal numbers of men and women are captured. The sample is calculated using the Cochran formula (1977) [32]; by taking 50% as a proportion of patients’ perception on quality of primary health care services) and a power of 80% allowing for an estimated error margin of 5%, the sample size obtained was 384 patients. To account for refusals, an additional 10% of the sample will be added (n = 38), making a total of 422 patients.

The qualitative samples and sample sizes for this component are as follows:

We will use purposive sampling technique, to select at least 8 participants for in-depth interviews as a sub-sample of exit interview questionnaire (for exiting patients) from each district, i.e., 4 participants from dispensaries and four participants from health centers, thus making a total of 64 patients. Also, a total of 4 health care workers will purposively be selected for in-depth interview per council, i.e., 2 participants from dispensaries and 2 participants from health centers, thus making a total of 32 in-depth interviews. The exact number of interviews will be based on saturation of information that is the sample could even be smaller or increase depending on saturation. The interviews will explore the process and experiential dimension of quality of care and will complement evidence emerging from the quantitative analysis.

Data collection tools and procedures. To understand the data use situation including their determinants (governance, technical, behavioral and context) and existing initiatives and the indicators related to quality of care at the health facility, we will rely on facility assessment tool, interviews with clients and a structured questionnaire to health care workers and an observation checklist. The tools aim at getting a holistic picture on changes in quality of care and related health outcomes overtime as the initiative is implemented or as a result of changes in time. The tools are described here under and are found in S2 File.

1. Facility assessment tool: We will use the star rating tool to understand the baseline situation in relation to quality of care of the health facilities. The star rating tool is a validated instrument that has been used to measure quality of care in the health facilities in Tanzania. We will measure the changes in indicators related health system building block indicators (financing, availability of health commodities, and motivation of human resource for health) that could reflect the effects of data use at health facility level.
2. Patient exit interview questionnaire: We will use a patient exit questionnaire to understand patient’s experiences and perceptions on the quality of care provided in health facilities. These questionnaires are also part of the star rating tool, i.e., we will adopt all the questionnaire in the star rating tool
   NB: The star rating tool consists of facility assessment and exit interview questions and has been used repeatedly to assess quality of health care services in Tanzania [33]. During the baseline assessment, we will also pilot and validate an organizational readiness tool geared to assess health workers readiness to implement data use interventions
3. Health workers questionnaire: We will use a health workers questionnaire to understand the situation in relation to data use and the determinants of data use at the facility
4. Observation checklist: We will use an observation checklist that aims at getting the reality in relation to data use tools, equipment and practices related to data use at the health facility
5. For the health outcomes: we will extract data from DHIS2 for selected indicators using a document review checklist/DHS2 data extraction checklist. The indicators include: Number of health facility deliveries, number of pregnant mothers completed 4th ANC visit, NC Total Pregnant Women (PW) tested, Number of pregnant women who tested positive for HIV (First Test), Proportion of pregnant women on ARV to prevent HIV transmission and Number of Maternal Deaths

Study component 2: Measuring implementation process of the data use initiatives. This component aims to establish data use initiative implementation processes as well as influencing (moderating) factors. This component will be implemented in the intervention councils to better understand the initiative’s implementation processes, thus unlocking the data use initiative’s black box by better knowing the initiative’s components and contextual impacts.

Sampling and sample sizes. Our sample for measuring the implementation process (process evaluation) will include implementers of the data use initiative at the council and health facility level. The participants will be implementing partners, health care mangers and health facility governing committee members (CHMT, HFGCs, health facility-in charges and frontline health workers). The fact that all CHMT members are guardians of data use at the council and its respective health facilities, they will all be included in the study (census) thus making a total of 60 CHMT members i.e., at least 15 members from each of the participating council. At least two members of HFGCs will be recruited in the study making a total of 80 HFGC members. A consecutive sample of health care workers from the participating health facilities with a maximum of five participants based on the assumption that the selected facilities should have a minimum of five health care workers, i.e., a total of 200 health workers will be recruited in the study. In addition, routine health facility files and documents will be read and summarized to fit the study context.

Data collection tools and procedures. The domains for the implementation process evaluation are reach, adoption, fidelity of implementation, acceptability and sustainability. Data will be collected from the following types of participants and documents; implementing partners, health care managers and health facility governing committee members (CHMT, HFGCs, health facility -in charges and frontline health workers) and health facility documents. Table 1 provides the details on the set of tools and their alignment to objectives and sources of data. In line with our mixed methods design, quantitative data will be gathered using; structured questionnaires, document review/ observation checklists, and semi-structured in-depth interviews (IDIs).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. The set of tools and their alignment to objectives and sources of data.

https://doi.org/10.1371/journal.pone.0303552.t001

The questionnaire for health workers will capture information related to socio-demographics, general knowledge on data use initiative, acceptability of the initiative to health workers, fidelity of implementation, sustainability of data use initiative and moderating factors. The questions in this tool will either be multiple choice, yes/no or Likert scale.

A structured observation checklist will be used to collect data on implementation of data use initiative. This will serve as triangulation of the information obtained from the health care workers. We will use semi structured instruments to collect data from the implementing partners, health care mangers at all levels and frontline health workers. The qualitative component aims at uncovering the participant’s experiences with data use and their expectations on the data use initiatives and its future maintenance at the health facility. All the tools to be used in this study are detailed in S2 File.