The 12 step implementation process for the electronic register

A 12-step implementation process for the electronic register was collaboratively developed by CIDER and the Anova Health Institute (Fig 2). Master training based on this 12-step implementation process aimed to structure and standardise implementation practices among the key implementers across all South African districts. This process is explained in detail:

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. The 12-step implementation process for the electronic register.

A 12-step implementation process for the electronic register was collaboratively developed by CIDER and the Anova Health Institute. Master training based on this 12-step implementation process aimed to structure and standardise implementation practices for the electronic register among the key implementers across all South African districts.

https://doi.org/10.1371/journal.pone.0127223.g002

The first step involves stakeholder meetings aimed at ART Programme Coordinators, primary health care (PHC) managers, pharmacists, as well as facility managers, ART nurses, doctors, and administrative clerks. During these meetings, the electronic register is introduced to staff members, its use along with that of the clinical stationery are discussed, whilst the requirements for roll-out including the timeline, equipment, staffing, and training needs are planned in detail. Facility visits are conducted to identify available resources such as computers, space in the facility, the number of ART patients remaining in care, the current data flow, day-to-day operational functions, previous record systems, and staffing. A facility evaluation report outlining the situation at each facility is developed to map out the recommended implementation.

Facilities are readied for roll-out, equipment (computer, memory stick, access to printer) is procured, and administrative clerks are deployed if required. M&E logistics are evaluated and adjusted at facility-level. All individuals involved are trained on use of the electronic register, general HIV and ART information, and their roles and responsibilities in implementing, reporting, and maintaining the register. Thereafter, current data from the paper-based register are captured onto the electronic register. That is, every patient who received ART at the facility, including those who defaulted on their ART, those who have died, and those who transferred-out to another facility, are captured. Once this process of back-capturing is completed, data clean-up is pursued. This important phase of the process requires administrative clerks to export data to Microsoft Excel and to perform specific checks to identify outliers and spurious reports in the data.

The subsequent phase of the implementation process, referred to as “post-implementation,” necessitates the “live” capturing of patient data directly from clinical stationery instead of the paper-based register. In routine practice, clinicians ensure that clinical stationery is completed and that patient folders are returned to the administrative clerk for data capturing prior to being filed. Administrative clerks are responsible for capturing data into the electronic register and for generating routine facility reports. These reports are signed off by facility managers, who maintain overall responsibility for programme data. Facility reports are sent to sub-district and district information management who ensure that data are captured on the District Health Information System. These individuals provide feedback on data quality to facilities. Facilities receive on-going training and support, including updates of newer software versions as well as technical support.

Ethics Statement

All study participants gave written consent to participate in a personal audio-recorded interview. The protocol and consent procedures were approved by the Health Research Ethics Committee, Faculty of Health Sciences, University of Cape Town and the Western Cape Provincial Research Health Committee (both in South Africa).

Ease of implementation of the electronic register

Respondents were asked to indicate their agreement with the statement, “the process of implementation of the electronic register was very difficult” on a scale of 1 to 5 (1 showing strong disagreement and 5 showing strong agreement). The majority of respondents did not find implementation of the electronic register difficult. The extent to which respondents perceived implementation to be difficult varied by respondent type and in two cases between respondents from the same facility. Reasons cited for positive experiences included: familiarity and experience with the paper register or with other electronic tools such as the TB register (ETR.Net) and a local paediatric ART register; receiving training on the electronic register; and the intensity of support from the partner implementer during facility preparation and roll-out.

The programme itself is not difficult, because we work on ETR.Net, [the electronic TB (Tuberculosis) register], and it is exactly the same.

[Respondent 17, Sub-district/District management]

I didn’t find anything difficult [about implementation]. I knew how to work with this [register] because we had the training before.

[Respondent 5, Administrative clerk, Facility B]

I would say [implementation] went relatively well here, because [the partner implementer] helped us with the roll-out and training on the electronic programme.

[Respondent 16, Sub-district/District management]

Someone who doesn’t have speed [with a computer] will naturally find [implementation] difficult.

[Respondent 13, Administrative clerk, Facility E]

The perceived value of an electronic M&E system

All twenty respondents thought that the electronic register had advantages over a paper-based M&E system. The perceived advantages of an electronic register positively influenced the ease with which the system was adopted for implementation.

I was willing [to have the register implemented] because I know electronic things work faster and are more accurate [than paper].

[Respondent 15, ART nurse/Facility manager, Facility E]

There were not really many hiccups [in terms of achieving buy-in] since [the electronic register] was introduced for roll-out or any resistance from management that I can think of. We realised the programme was going to help us.

[Respondent 16, Sub-district/District management]

All respondents scored positive responses on a Likert-scale when asked whether the implementation of the electronic register had a positive impact on their work. In reflecting on their experiences of using the electronic register post-implementation, perceived advantages of the register varied by respondent type and demonstrated how the system held distinct benefits for each work role. Administrative clerks enjoyed being able to produce faster and more accurate monthly and quarterly facility reports:

I struggled with [aggregation of] the quarterly report [from the paper register] because I had to count everything. [The electronic register] produces the list automatically. It is just a click of the button.

[Respondent 2, Administrative clerk, Facility A]

Nurses explained that they had increased access to patient data for day-to-day patient-specific queries (e.g. blood results, last facility visit date, drug regimen). Facility managers noted that their knowledge of the HIV-positive cohort attending their facility has improved, while sub-district and district respondents specified that they had immediate access to facility-based data.

I update the dispatches myself. [The electronic register] has made it much easier to zoom in on the ART programme, to equip and empower myself. It has made it easier for me to plan, and to give answers.

[Respondent 16, Sub-district/District management]

Positive experiences with electronic ART data were tempered by mistrust of data at facilities that had poor quality paper registers as the source data, administrative clerks who were new to the ART programme, or where the regularity of data capturing post-implementation was a concern. Such concerns were only expressed by facility managers and sub-district respondents, who in some cases were uncertain of administrative clerks’ ability to capture medical information accurately, or were anxious regarding the maintenance of ART data post-implementation.

The system is good if [the data] are just entered correctly. That is one of the gaps, because if it is not entered correctly, then I cannot get the correct information […] We still have the medical background. I know about CD4s and those things. Those children [i.e., the administrative clerks] sometimes know nothing about these things.

[Respondent 12, Facility manager, Facility D]

Importance of stakeholder engagement

The buy-in meeting was perceived to be the gateway to implementation. Sub-district and district respondents noted that drawing in key role players at facility-level, i.e., the facility manager, ART nurses, and administrative clerks, would enable the implementation process.

[Buy-in] is very important. In our sub-district we immediately pulled in our Senior Administrative Officer, all the administrative clerks in the clinics, they were part of the meeting when [the register was introduced for roll-out]. You have to do a bottoms-up approach, first do it in a sub-district, let people see that it works and then you go wider. Let a small sub-district show you it can work and win over the [district] Director.

[Respondent 18, Sub-district/District management]

ART Programme Coordinators generally do not have a line management function over the clinics’ administrative clerks and nursing staff. [They] are here only to provide support to the programme, because your operational managers are the line managers. So [facility-level staff] must be involved from the start and be part of the story [in order for implementation to be successful].

[Respondent 16, Sub-district/District management]

Buy-in for implementation was exhibited in various ways by facility-level staff, such as through the readiness with which a facility manager freed her staff for training, consideration for the administrative clerk during the back-capturing phase, eagerness with which space was made available to accommodate the system’s requirements, openness of staff to receive training, or watchfulness over the patient folder flow so that data could be captured timeously.

In addition to overall willingness, support by the partner implementer was consistently emphasised by all respondent types as facilitating implementation in terms of tangible and intangible system requirements. This included:

• Provision of resources such as computers and administrative clerks during the back-capturing step;
• Training and mentoring on the electronic register and correct use of clinical stationery; and
• Training administrative clerks on basic HIV and ART knowledge to facilitate data capturing.

Support from the partner implementer enabled role players to overcome potential obstacles to implementation. The on-going support provided by the partner implementer was important to encourage respondents to overcome challenges during implementation and to ensure that implementation would continue.

The thing that made [implementation] the easiest for me was [the partner implementer], his expertise about [the electronic register], he was hands-on. [The partner implementer] trained everybody [on the electronic register]. It was not like train-the-trainer; the trainer was there. If I just tried to tell him that something is not going to work then he told me, “Come on!” and he makes it work. It was one of the nicest projects [I did] in the [sub-district].

[Respondent 18, Sub-district/District management]

Influence of a data champion

Administrative clerks’ experience with ART programme M&E, level of computer literacy, and personal self-efficacy were found to develop the confidence of others in the facility for implementation. Where such highly-motivated individuals or champions were present during implementation, respondents communicated positive experiences of implementation.

I think [the administrative clerk] took to leadership. He and I sat together for an hour, just so that he could show me [on the register]. Then I realised that [the electronic register] is similar to the ETR.Net (the TB register). When he was here [the electronic register] was implemented quite well. It is because of the training that he had already received. He was exposed to [the ART programme]; he had the experience.

[Respondent 7, Facility manager, Facility B]

Such positive experiences during implementation carried over into the post-implementation context as clinical staff trusted the data champion to take responsibility for the maintenance of the programme.

I think [implementation] has been successful because the administrative clerk has been trained very well on how to [capture data]. And she has made a point to do it on a daily basis and in an exact way, so the [data] has been up to date up until now.

[Respondent 9, ART nurse, Facility C]

Conversely, respondents at facilities without a data champion—either during implementation or post-implementation—exhibited fewer positive experiences of implementation and showed uncertainty over the maintenance of the data for the ART programme.

[Our current administrative clerk] used to be an administrative clerk at [another clinic]. She was taught to pull folders at [reception] without a computer. She didn’t have the experience. See, she wasn’t exposed to ARV (antiretroviral) information. She learnt on [the electronic register] here [at the clinic]. They are busy training her now, and eventually there will be an improvement [in how the data is managed].

[Respondent 7, Facility manager, Facility B]

Operational and logistical factors

Several operational and logistical factors affected implementation. Respondents singled out the back-capturing process as the most challenging factor; other important factors were availability of resources required for implementation, and the level of integration of ART services in the facility.

Workload and role clarity

Despite having favourable factors to suggest successful implementation, the roll-out process was hampered at Facility E due to a high workload and perceived lack of leadership. Back-capturing took four to five months to complete given a high workload and confusion regarding responsibilities of administrative clerks and the procedure for reporting ART data during this time. Respondents suggested that clearer communication regarding the above would have helped implementation.

[Back-capturing] had taken months and months [to be completed] and every time we get to meetings it is only [this facility] which is still in [back-capturing] mode. No one asked me [to help], and I just went on [with my work]. But we couldn’t send in our statistics while we were back-capturing. And that is perhaps where it needs clarity—whether the senior administrative clerk is also a data capturer. Who takes responsibility for the stats?

[Respondent 13, Administrative clerk, Facility E]

Importance of integrating the electronic register with routine facility M&E

Respondents consistently raised the importance of integrating the electronic register with routine M&E structures. Post-implementation, respondents noted that there was an occasional operational challenge with the maintenance of ART data as administrative clerks who worked primarily with PHC data did not take accountability or ownership of ART data as routine facility data. This situation created complications for the maintenance of ART data when the designated ART administrative clerk was absent. In such instances, administrative clerks reported that there would be a backlog of ART information to be captured, or that uncaptured folders would be refiled, which could compromise data quality.

Some administrative clerks stressed their attempts to provide in-service training for their PHC counterparts who had not received training on the electronic register, so as to overcome the issue of data maintenance by other clerks. They maintained that their efforts were often met with resistance.

According to [the PHC administrative clerks], I am the administrative clerk for the ARV-clinic so I can just go on capturing the data for [the electronic register]. I don’t know if they will also [capture the data], they are not interested. I don’t know if they are scared of [the electronic register] or what.

[Respondent 8, Administrative clerk, Facility C]

Sub-district and district managers explained this situation by emphasising that the lack of ownership of ART data in the facilities was part of a larger issue dating back to the vertical roll-out of the ART programme and vertical reporting systems and data flows for ART M&E.

[The PHC administrative clerks] capture all the other [facility] data, but [the electronic register] is almost a mystical something which is apart as ARVs are. I think part of the problem is that NIMART (nurse-initiated management of ART) was not rolled-out correctly, so the ownership of the ARV patients does not lie entirely with the [PHC] clinic yet.

[Respondent 19, Sub-district/District management]

In the past ARVs were quite separate. When the call for integration came [staff] at facility-level started to realise, the operational manager, “If [ART data] comes to me to look at, to the sub-district, to the information people, to the PHC manager, then [ART patients] are our patients.” So I would think that with [the electronic register] and the integration of the ART programme, [the register] came at the right time to bring it all together.

[Respondent 16, Sub-district/District management]

Respondents pointed out that training the entire facility staff component on the electronic register would facilitate the maintenance of the data post-implementation. Administrative clerks in particular noted that drawing in various facility staff would inculcate an understanding of the importance of their role in ensuring good quality ART data, for example, through the correct completion of ART clinical stationery, facilitating folder flow, and enabling consistent data capturing.

I think in future they have to see to it that the [PHC] administrative clerks are on par with [the electronic register]. Not just to have a one-sided focus and say that it is the ARV data capturer’s problem. Because what will we do if the ARV data capturer is sick one day? Who will take over?

[Respondent 7, Facility manager, Facility B]

[The administrative clerks] have good intentions to capture all folders every day, but it doesn’t happen that way in practice. So the workload [of capturing ART data] is a big thing that was added, but there were no extra hands to do it. The administrative clerks are quite busy—you don’t see an administrative clerk standing around doing nothing […] And it is a new thing too, because they are not computer literate and I think for them it is a bit of a stressor. It is just that one needs more training sessions, follow-up training sessions.

[Respondent 19, Sub-district/District management]

Node 1: Adoption of a standardised electronic ART M&E register

The first node covers the dissemination and adoption of the electronic register for implementation: the intervention characteristics, outer setting, and process are the most appropriate domains to describe this part of implementation. External policy & incentives, or leadership from provincial Department of Health, is related to the outer setting and was an important catalyst for implementation as it mobilised key stakeholders in ART M&E to vie for roll-out of the tool in districts in the Western Cape. Such endorsement of the electronic register, what Greenhalgh et al. [9] term “political ‘must-dos,’” facilitated buy-in for the intervention.

A key dynamic in the dissemination of the electronic register and a precursor to its implementation at facility-level was the relative advantage of the register as well as engagement (in this case from the partner implementer). This is in line with Greenhalgh et al. [9] who maintain that relative advantage is a requirement for adoption or buy-in from stakeholders. The authors explain that despite the relative advantage or strong evidence in support of an intervention, buy-in is not automatic but requires a process of engagement to clarify the relative advantage of the intervention to potential recipients. This involves dissemination of information regarding the intervention, development of an implementation plan, and support of those involved in implementation in carrying out the plan.

Our findings suggest that as a simple electronic bridging solution to the paper-based register, the relative advantage of the electronic register was immediately apparent to district and sub-district stakeholders and facility-level staff. The electronic register offered simple digitisation of ART data, making it compact, allowing health care workers “click-of-a-button” access to patient-level data, faster and easier reporting, and increased accessibility of data for facility, sub-district, and district management, while requiring comparatively few resources. The CFIR refers to individuals who “formally influence or facilitate intervention decisions in a desirable direction” as external change agents [7]. In this instance, buy-in was driven by a partner implementer at sub-district and district level. The longstanding relationship between the partner implementer and the Department of Health in the Cape Winelands arguably also helped the buy-in process, as did the similarity in backgrounds of those involved in implementation, as suggested by Greenhalgh et al. [9]. Although such a driver took the form of an outside organisation, we suggest that engagement need not come from an external organisation; an internal motivated individual or individuals could achieve the same dynamism.

In summary, the relative advantage of an intervention paired with engagement from enthusiastic individual(s) who take responsibility to drive implementation are critical for successful adoption of an intervention. In this case, policy & incentives played a role in setting the agenda for roll-out yet it cannot substitute the power of an intervention that makes sense to those involved, and key individuals who help to sell the intervention to those potentially involved in implementation.

Node 2: Implementation at facility-level—a complexity versus compatibility balance

The second node refers to implementation at facility-level. Most of the ten characteristics we found influenced implementation of the electronic register intersect at this node which hints at the complicated nature of implementation at ground-level. The key domains are intervention characteristics, inner setting, individuals involved, and process. Interaction between these domains is multifaceted; as Damschroder et al. [7] suggest, “without adaptation, interventions usually come to a setting as a poor fit, resisted by individuals who will be affected by the intervention, and requiring an active process to engage individuals in order to accomplish implementation.” This statement encapsulates the four domains highlighted by our findings. At ground-level each of these components of implementation are at their most complex. Abbott et al. [8] describe health care facilities as “complex and adaptive systems” where “methods that work in one organisation or location may fail in another,” emphasising that implementation is context-dependent. Moreover, Greenhalgh et al. [9] maintain that implementation is social in so far as it involves individuals. The authors explain that

People are not passive recipients of innovations […] They seek innovations, experiment with them, evaluate them, find (or fail to find) meaning in them, develop feelings (positive or negative) about them, challenge them, worry about them, complain about them, “work around” them, gain experience with them, modify them to fit particular tasks, and try to improve or redesign them—often through dialogue with other users.

Intervention characteristics—relative advantage and complexity—were particularly important for facility-level implementation as they influenced the receptivity of the inner setting for implementation. Relative advantage facilitated buy-in of facility-level staff, especially in large mature facilities where paper-based registers were becoming more and more cumbersome. Under such circumstances there was an added tension for change (as opposed to younger or ART-naïve facilities). We found that complexity of the intervention, which comprises amongst others the perceived difficulty, duration, and number of steps required to implement, was not specific to the intervention. Instead, complexity was influenced by factors in the inner setting and characteristics of individuals involved in implementation. Osler et al.[5] explain that

with each higher tier comes added complexity and the need for additional support. The second-tier solutions [such as the electronic register] require computer literate staff, computer availability, and software and hardware support, in addition to the training and protected staff time that are common requirements of all tiers.

Key inner setting factors included the quality of paper registers, workload (which related to the maturity of ART services), resources (material, human, physical space, training), level of ART service integration, as well as experience with ART programme M&E, level of computer literacy, and personal self-efficacy of individuals involved. These factors informed the implementation climate (in terms of the intervention’s compatibility with existing workflows and systems), the readiness for implementation (in terms of available resources and access to knowledge and information) of facilities, and the self-efficacy of individuals involved to achieve successful implementation. Implementation was notably easier in facilities which rated favourably in these characteristics and constructs.

Even in facilities where most system requirements were in place prior to implementation, interaction between the intervention, inner setting, and individuals involved were not necessarily favourable. As with node 1 implementation required an active process of engagement. At facility-level, this relied on two actors namely the data champion and external change agent. These key role-players facilitated the complexity of the intervention, compatibility and readiness for implementation in the inner setting, and self-efficacy of individuals involved.

Data champions positively influenced implementation by making themselves accountable for the success of implementation, holding involved others accountable, and developing the confidence of less-involved others by drawing on their personal self-efficacy. Data champions by definition exhibited pride in their work and an enthusiasm to learn, which could overcome initial shortcomings in skill (computer literacy and experience with the ART programme M&E for instance); the latter capacities could be developed through training and support. In terms of optimal implementation, data champions are a necessity as they are able to arbitrate and potentially overcome barriers to implementation as they arise at ground-level.

Whereas data champions were important for operations at facility-level, the external change agent played a critical role in achieving buy-in of facility-level staff, facilitating the implementation climate (compatibility), readiness for implementation (in terms of available resources and access to knowledge & information), and developing the self-efficacy of individuals involved. As evidenced in our interviews, such an individual who drives implementation at various levels—district, sub-district, or facility—is invaluable in maintaining the momentum for implementation and responding to needs as they arise, be they related to individual skills development or infrastructural requirements for implementation.

In summary, implementation at facility-level is complex as it entertains various components of implementation which are of themselves complex: the intervention, inner setting, and individuals involved. To achieve successful implementation at facility-level, the interaction between these components requires supervision and facilitation by individual drivers of implementation. In our five facility contexts, data champions and an external change agent took on this role in their varying capacities to overcome challenges and barriers to implementation. Yet, despite less desirable factors and the absence of a data champion in some facilities, our findings show that successful implementation (in terms of summative outcomes, i.e., regular reporting) can still be achieved.

Based on the preceding discussion of implementation at the two nodes, i.e., at the adoption of the electronic register for roll-out, and at implementation at facility-level, we found that relative advantage, an intervention characteristic construct, and engagement, a process construct, were the only constructs to overlap between nodes. This suggests that these constructs are critical aspects of implementation which have the capacity to influence outcomes of implementation at both adoption and coalface stages of implementation.

Strengths and limitations of the study

We identified a number of strengths and limitations in our work. The primary strength was the qualitative nature of the methodology. The nuance and complexity of implementing any new intervention, at least in this setting, was best measured through in-depth interviews. For the scaled-answer questions a respondent may indicate one thing, e.g. that the implementation process was not difficult, while during in-depth interviews various challenges would be revealed. Also as Abbott et al. [8] identify, implementation science is in its infancy and qualitative approaches are ideal to collect the various constructs and casual pathways for which quantitative approaches can be used in future. One of Abbott et al.’s [8] identified best practices include to “understand how the multiple levels of complex interventions intersect and how they relate to the intervention. We were well-aligned to this as we interviewed with multiple levels of the District Health System (facility, sub-district, and district level) as well as interviewing across cadres within the facility (administrative clerk, nurse, and facility manager).

Limitations of the research include limited generalisability across South Africa, methodological limitations relating to the facilities and respondents included in our study sample, and potential researcher and respondent bias.

South Africa is a diverse and complex nation, and this extends to its public sector health facilities. As such, our findings may not uniformly apply to South Africa’s fifty two districts. Findings and recommendations should be considered and tailored to each district’s context, especially as the Cape Winelands district had dedicated administrative clerks for the ART programme in each facility at the time of implementation; this is not the case everywhere. Yet, we consider our sample of facilities and respondents to sufficiently represent implementation in this context. The facilities selected include diverse facility characteristics, and we reached saturation in our interviews. It is reassuring to have found that the characteristics and constructs highlighted as important to implementation, whether positively or negatively applied, are shared across the five facilities. This gives weight to our ultimate findings on factors which influence implementation.

In terms of researcher bias, three of the interviewers and analysts were loosely affiliated with the partner implementer which may have affected their pure objectivity. To clarify, two researchers were permanently employed by Anova Health Institute, another was appointed on a contractual basis to work on this project specifically, and the last person was affiliated with a local university and not with the organisation. It is noteworthy that the partner implementer would not gain from a biased representation of implementation. Instead, the partner implementer and those interviewers and analysts loosely associated, had a vested interest in unpacking and identifying gaps in implementation in this context so as to inform and guide future implementation efforts. This is also the reason for engaging theory on implementation more broadly. This goal was a driving force for the research, and the interviewers and analysts took great care to remain objective in analysis and interpretation through systematic coding of themes and regular discussion on the interpretation of findings through an implementation framework.

Another methodological matter is whether respondents may have provided desirable answers since some of the researchers were allied with the Anova Health Institute. This is a consideration in all research but one of particular pertinence in this study given the longstanding relationship between the partner implementer which headed the evaluation and the Department of Health in the Cape Winelands. Such immersion and familiarity with a study context has both pros and cons. In this case, we believe that there were some benefits to the familiarity in terms of trust and a deeper understanding of the local cultural competency. Our findings reflect nuanced views of implementation across the five facility-contexts, including both positive and negative experiences of implementation. Although desirable answers could have been given in some instances, these were arguably overshadowed by the varying perspectives we gathered from interviewees across different levels of the health system in the district. Another consideration is that the study was conducted a year after roll-out, requiring respondents to reflect on the implementation process some time after it had been completed, i.e., while working in a post-implementation context. This could introduce recall bias. Lastly, there was a data audit component of this research, but as it only established a baseline of data quality in the electronic register, those data could not be triangulated to understand the impact of implementation on data quality. Although beyond the scope of this paper, we acknowledge that data quality is critical to successful implementation as these data inform evidence-based decision-making in the ART programme. Kaposhi et al. [10] outline this crucial aspect of ART M&E using the Eastern Cape province of South Africa as case study. To mitigate the issue of data quality, the Western Cape Department of Health introduced an ART M&E standard operating procedure in November 2013. Future evaluations of the electronic register should investigate the issue of data quality, implementation of the standard operating procedure, and make recommendations for the maintenance and improvement of data.

Our findings suggest a more broad evaluation of implementation characteristics that could provide a greater understanding of which domains, constructs, and interactions result in more rapid and lasting implementation of new interventions.