Association between Health Systems Performance and Treatment Outcomes in Patients Co-Infected with MDR-TB and HIV in KwaZulu-Natal, South Africa: Implications for TB Programmes

Objective To improve the treatment of MDR-TB and HIV co-infected patients, we investigated the relationship between health system performance and patient treatment outcomes at 4 decentralised MDR-TB sites. Methods In this mixed methods case study which included prospective comparative data, we measured health system performance using a framework of domains comprising key health service components. Using Pearson Product Moment Correlation coefficients we quantified the direction and magnitude of the association between health system performance and MDR-TB treatment outcomes. Qualitative data from participant observation and interviews analysed using systematic text condensation (STC) complemented our quantitative findings. Findings We found significant differences in treatment outcomes across the sites with successful outcomes varying from 72% at Site 1 to 52% at Site 4 (p<0.01). Health systems performance scores also varied considerably across the sites. Our findings suggest there is a correlation between treatment outcomes and overall health system performance which is significant (r = 0.99, p<0.01), with Site 1 having the highest number of successful treatment outcomes and the highest health system performance. Although the ‘integration’ domain, which measured integration of MDR-TB services into existing services appeared to have the strongest association with successful treatment outcomes (r = 0.99, p<0.01), qualitative data indicated that the ‘context’ domain influenced the other domains. Conclusion We suggest that there is an association between treatment outcomes and health system performance. The chance of treatment success is greater if decentralised MDR-TB services are integrated into existing services. To optimise successful treatment outcomes, regular monitoring and support are needed at a district, facility and individual level to ensure the local context is supportive of new programmes and implementation is according to guidelines.


Introduction
In KwaZulu-Natal many patients have multidrug-resistant tuberculosis (MDR-TB), defined as TB resistant to isoniazid and rifampicin. [1] Fuelled by concomitant hyper-endemic TB and HIV, KwaZulu-Natal has one of the largest drug-resistant TB epidemics in the world. [1,2] Management of MDR-TB is complex, and different interlinked health service components influence each other affecting MDR-TB treatment outcomes. To provide treatment for patients closer to their homes, a decentralised model of treatment was initiated at four sites in 2009. To guide implementation of the MDR-TB programme in the four sites, provincial treatment guidelines were developed and distributed.
Successful implementation of any MDR-TB programme is dependent on different health system components functioning together to support effective service delivery, but there is limited evidence on how programmes interact with health systems and which factors enable or hinder this interaction. [3][4][5] Furthermore, although much has been written about the patient and disease characteristics that impact on MDR-TB treatment outcomes (TOs), there is little evidence of the impact of health system performance on TOs. [6] Negative health care worker attitudes, drug stock-outs, large cohort size and limited resources with which to trace defaulters are some of the factors which have been described as contributing to poor TOs [7][8][9][10].
To evaluate implementation of the decentralised MDR-TB programme we undertook a study comparing the effectiveness of decentralised care for MDR-TB patients with care in a centralised setting. [11,12] During the comparison we noted that the MDR-TB programme was implemented differently at each decentralised site. Furthermore, treatment outcomes varied considerably between the sites, with some sites performing better than the centralised hospital and others worse. To better understand the diverse implementation and the subsequent varying treatment outcomes, we performed an analysis of health systems factors. We hypothesized, that treatment outcome was associated with local health system factors and that by investigating the association, we could identify those factors critical to successful treatment outcomes.
To determine that association, ''health system performance'' (HSP) was defined as one variable. For each site, HSP was the result of a composite assessment of four performance domains: context, integration, mechanism and output. [13,14] Treatment outcome (TO) at each site, our second variable, was based on the site's overall treatment outcomes.
In this exploratory prospective mixed methods health systems comparative case study of the four sites, we addressed the following research questions. (1) Is there an association between treatment outcome (TO) and health system performance (HSP)? (2) Which domains of health system performance are associated with successful TOs?

Ethics Statement
The study protocol was approved by the University of KwaZulu-Natal Biomedical Research Ethics Committee (Ref: BF052/09), and by the KwaZulu-Natal Department of Health. Only secondary data, the data routinely collected by health workers for clinical care was used in this study. To protect patient confidentiality and anonymity the data bases were de-identified and access strictly limited. Informed consent was waived by the ethics committee, since all patient data used were previously collected during the course of routine medical care and did not pose any additional risks to the patients.

Study Design
This mixed methods case study of four decentralized MDR-TB sites between 1 July 2008 and 30 June 2012 was a prospective health systems study.

Study Population
The Provincial TB directorate identified four sites, the cases for our case study, for implementation of the decentralised MDR-TB programme. These sites were purposively selected in areas where large numbers of patients with MDR-TB were being diagnosed. Although distributed widely across the province the infrastructure and socio-economic status of the populations in these districts was similar. These sites together with their health care workers and managers were included in the study, as were district-level managers involved in MDR-TB management from the districts where the sites were based.
All patients from the four decentralised sites with a culture confirmed diagnosis of MDR-TB, age >18 years, and who commenced treatment between I July 2008 and 30 June 2010, were included in the study. No data was collected after October 1, 2012. Inclusion criteria for the comparison study required that patients reside within the catchment area of the site. Patients receiving care at more than one site were excluded, as were patients who had MDR-TB with additional resistance to amikacin, kanamycin, capreomycin or any fluoroquinolone.

Data Collection
We reviewed medical records to collect patient-related demographic, clinical, pharmaceutical and laboratory data. All data, was collected prospectively, prior to knowledge of patient treatment outcomes. Health system data was collected from different components of the health system -laboratory, pharmaceutical and transport services and human resources -using existing records and databases, structured questionnaires, observation and interviews. As differences between the sites and complexities emerged, an iterative approach enabled us to identify new health system data required and develop appropriate data collection methodologies.
Quantitative data was complemented by qualitative data obtained through participant observation and discussions with staff. Over the four year study period each site was visited monthly for a day (ML). During each visit data from each health system component was collected (see Table S1), the functioning of the MDR-TB unit observed and informal discussions held with the nurse-in-charge of the MDR-TB unit, the clinician responsible for MDR-TB and the hospital pharmacist. Through a process of ongoing reflection, feedback and discussion with facility and district level staff problems were investigated to determine their origin and cause and possible solutions identified. Field notes detailing the visit and documenting observations and discussions with staff were written up after returning from the site. Notes were also made of concerns, opinions and issues which needed follow up.

Variable Definitions
In defining the HSP variable we adapted a conceptual framework, which had been validated both internationally and in our setting, [13,14] and identified key domains of health system performance -context, integration, mechanism (comprised of support services and human resources) and output (Table 1). To measure HSP we identified health system factors which would affect system performance. Concomitant indicators with which to measure the impact of these factors on each domain were then identified and defined (Table 1).
To measure indicators, data collected at each site was scored by an investigator (ML) and the nurse-in-charge of the MDR-TB unit ( Table 2). Where appropriate, the score incorporated the date of implementation (earlier was scored higher than later) and consistency (greater consistency scored higher than partial or lower consistency). A total score for each domain was calculated and converted where necessary, so that each domain was equally weighted. The sum of the domain scores provided an overall HSP score for each site with the maximum possible score for any one site being 160. Annually, over the four year study period a HSP score per site was calculated. At the end of the study an average score for each indicator, health system factor and domain was calculated for each site (Table 3). Similar scoring methodologies have been used to measure TB and HIV integration at primary level clinics in South Africa. [15,16].
Treatment outcomes (TOs) of patients were determined at the end of treatment, according to definitions developed by the WHO (Table 4), [17,18] based on patient data indicating successful treatment (patient was cured or completed treatment) or unsuccessful treatment (failure to respond to treatment, default or death). The overall percentage of these outcomes was calculated for each of the four sites.

Analysis
Data analyses were conducted using SAS version 9.2 (SAS Institute Inc., Cary, NC, USA). Differences in outcomes across sites were compared using binomial regression. Statistical significance was set at alpha = 0.05. Pearson Product Moment Correlation coefficients were used to quantify the direction and magnitude of association between HSP scores and successful TO by domain.
Qualitative data was explored and analysed in light of the introduction of a new programme. Using systematic text condensation (STC) the field notes were read and possible themes identified. [19] Through decontextualisation and a process of reflection on commonalities and differences, themes were classified as codes. Condensation provided meaning to the codes which were finally synthesised into our conceptual domains.

Treatment Outcomes and HSP Scores
TOs of the 736 patients treated at the 4 decentralised sites are tabulated in Table 5. Across the four sites, 76% of all patients were co-infected with HIV. Overall, 58% of patients at the decentralised sites had successful TOs (cured and/or completed). However, there were differences in treatment outcomes across the sites. Successful TOs varied from a high of 72% at Site 1 to a low of 51.7% at Site 4 (p,0.01) and Site 3 and Site 4 had significantly higher default and death rates respectively (p,0.01). A detailed comparison of patient characteristics and TOs has been reported. [12]. Overall HSP scores varied across the sites with Site 1 having the highest HSP score (132 out of a possible 160) and Site 4 the lowest score (59 out of a possible 160) ( Figure 1). Site 1 scored highest score in all domains, achieving the maximum score of 40 in the domain 'integration' compared to Site 2 (score = 24), Site 3 (score = 19), and Site 4 (score = 15). In contrast, Site 4 scored the lowest in three of the four domains with 'context' being the sole exception.

Analysis by Domains
Context domain scores varied from a high of 29 at Site 1 to a low of 4 at Site 3 ( Figure 1). As a consequence of regular on-site support visits by the district TB coordinator staff and facility manager support, staff at Site 1 felt supported by the district and local hospital, in contrast to staff at the other sites (Table 3: Context domain: District and facility level support).
'Whenever we have a problem we phone the district TB co-ordinator. She is strict with us, but is also helpful.' (Interview: Site 1: Nurse-in-charge of MDR-TB outpatients clinic) 'The district TB co-ordinator came to the opening of this MDR-TB unit……But since then has never been near. Him and the hospital managers…….they don't even know where the unit is.' 'The hospital managers help with sorting out problems? Never, not one!' Additional mobile clinics for the MDR-TB programme were identified by the district in the first year of the programme (2008). The district mobile services were re-organised so that each mobile provided services for all TB, HIV and MDR-TB patients in a smaller geographical area than that in which mobiles had previously operated attending to TB and HIV patients. From the mechanism domain a number of human resource issues affecting the functioning of sites 2-4 emerged. The stability of the health services was affected by the common hospital practice of the rotation of front-line health workers through the different clinical disciplines -a regular practice at sites 2 and 3. At these sites key clinical staff were rotated every three months through the different clinical disciplines, including the MDR-TB unit, leading to low scores for knowledge about MDR-TB, and poor stability and consistency in the services (Table 3). At Site 1, as two staff members were on extended sick leave, the site had a low score for availability of staff. However, this was offset by other HR factorsthe stability and knowledge of remaining staff together with support for these staff.  The nurse-in-charge of the MDR-TB outpatients was rotated every three months. The appointment systems were not functional, as she didn't know the patients and was unaware if they missed appointments. Consequently, this site had a significantly higher default rate than the other sites (p,0.01) ( Table 3). Consistency of services was undermined by a second common human resource practice -the appointment of managers in 'acting' positions. Site 4 scored a 0 for managerial support (Table 3).
I took up a problem that had emerged with the 'acting' head of Site 4 MDR-TB unit. From the output domain quality of care varied across the four sites and inadequate clinical skills together, and poor adherence to clinical guidelines contributed to fewer successful TOs at Sites 2, 3 and 4 (Table 3).
'I am only working in the MDR-TB unit for 3 months. I haven't seen any guidelines. I follow what was done before. Dr. X who worked here before is around. I haven't spoken to him. He is busy, and so am I.'

Association between TOs and HSP
We found a correlation between successful TOs and total HSP score (r = 0.99, p,0.01) (Table 6, Figure 2). Quantitative data analysis suggested that the domain 'integration', had the strongest association with successful treatment outcomes (r = 0.99, p,0.01) ( Table 6). However, qualitative data indicated that issues within the domain 'context' influenced both programme implementation and the 'integration' domain.

Discussion
Our exploratory study suggests there is an association between treatment outcomes (TOs) and Health System Performance (HSP). We found this association to be significant (r = 0.99, p,0.01), with Site 1 having the highest number of successful TOs and the highest total HSP score. Conversely, Site 4 had the lowest number of successful TOs and the lowest HSP score. In addition, our study suggests that the 'context' and 'integration' domains had the strongest association with successful TOs.
District level ownership and leadership enabled re-organisation and realignment of services at Site 1, the 'integration' domain. Different health system components, such as pharmaceutical and transport services, were incorporated into the new programme and, the inclusion of local key personnel with grounded clinical Table 4. Treatment outcome definitions*.

Treatment outcome Definitions
Cure Cure was defined as completion of treatment and $5 consecutive negative culture results in the final 12 months of treatment.
Treatment completion Treatment completion referred to completion of therapy but without bacteriologic documentation of cure.
Treatment success Treatment success has been defined as the percentage of patients in whom the treatment outcome was either cured or completed. That is, ''% successful = no. of patients cured+no. of patients completed treatment/Total no. initiated treatment6100''.
Treatment failure Treatment failure was defined as having more than one positive culture in the final 12 months of therapy, or if any one of the final three cultures was positive, or if more than one drug in the treatment regimen was replaced, or if treatment was terminated due to adverse events or no clinical improvement.

Default
Default was defined as an interruption in treatment for $2 consecutive months for any reason.

Death
Death was defined as all-cause mortality during MDR-TB treatment.
Unsuccessful treatment Unsuccessful treatment outcome has been defined as the percentage of patients in whom the treatment outcome was died, defaulted, or failed treatment.
Transferred out Transferred out: A patient with MDR-TB who was transferred to another reporting and recording unit a year after study-enrolment whose treatment outcome is unknown.
*Treatment outcome definitions used are WHO definitions for the management of MDR-TB. [17,18] doi:10.1371/journal.pone.0094016.t004 Table 5. Treatment outcomes of patients with MDR-TB treated at 4 decentralised sites in KwaZulu-Natal, South Africa*. experience and knowledge of the local situation resulted in the development of a model of care which built on available strengths and was appropriate for local needs. [20][21][22] For front line health workers and facility managers at Site 1, district-level ownership translated into regular on-site support visits by the district TB coordinator. These visits provided encouragement and supervision, and aided in the resolution of site-level problems, thereby enabling health care workers to deal with emerging difficulties instead of becoming 'mired in inertia'. [23] In addition, the visits led to increased accountability and a commitment to patient care resulting in improved adherence and a higher number of successful TOs. In contrast to Site 1, district and facility managers at Sites 2, 3 and 4 in failing to own the MDR-TB problem, underestimated the realignments and changes necessary for the health system. This led to partial implementation of the programme, limited managerial and support services support, delays in the re-organisation of the human resources and re-allocation of vehicles and poor integration of the MDR-TB programme into existing PHC services. Furthermore, as staff at Sites 2, 3 and 4 were not visited regularly they felt unsupported and unvalued, were unmotivated, and lacked commitment to their work and their patients, resulting in fewer successful TOs. Other authors support this interpretation; they have shown that 'context' has a high impact on the capacity of health services to co-ordinate and support effective service delivery. [24][25][26] In a study evaluating the HIV programme in Russia, Tkatchenko-Schmidt et al [27] described the impact of leadership, ownership and support on staff motivation and performance. A systematic review which examined the relationship between nursing leadership and patient outcomes found a significant association between positive leadership and increased patient satisfaction and reduced adverse events. [28] Other studies have documented the effect of political and managerial leadership on health system functioning and the increased likelihood of unsuccessful TOs when health systems are dysfunctional. [26,[29][30][31][32].
Much has been written about the need for and importance of vertical health programmes strengthening health systems. [33,34] In our study, district level leadership at Site 1 integrated the new decentralised MDR-TB vertical programme into the existing district health services. The re-alignment and re-organisation of the services enabled the district health system to benefit from and be strengthened by the introduction of a new programme. In contrast, at the other sites, the new programme with its additional resources were not integrated into horizontal service delivery, and the existing services neither strengthened nor capacitated. In addition, the integration of the MDR-TB and HIV programmes at Site 1 would have contributed to successful TOs, as the role of ART in the successful treatment of co-infected patients is well documented. [35,36].
Inadequate clinical skills together with delayed implementation of changes to clinical guidelines contributed to Sites 2, 3 and 4 scoring poorly in the quality of care component of the domain 'output' and achieving fewer successful TOs. In a recent article on new TB diagnostics, the difficulties in implementing new guidelines or changes to guidelines are described. [37] In this article the authors emphasise that even small changes in guidelines or algorithms are a major undertaking in a national TB programme. Given recent advances in TB diagnostics and therapy and possible changes to algorithms and regimens, national TB programmes need to be cognisant of the complexity of change. Up to date guidelines must be available at facilities and in our study guidelines were consistently available at Site 1 only (Table 3).
Other studies have documented the negative impact of unavailable guidelines and protocols on quality of care. [38,39].
To ensure that the most up-to-date guidelines and regimens are implemented, regular on-going training, support and supervision at the decentralised sites is necessary. Given that the sites are scattered throughout a large province, different models for providing ongoing training, such as electronic or in-service, need Figure 2. Association between successful treatment outcomes and total health systems performance score at 4 decentralised MDR-TB sites. This figure shows the association between successful treatment outcomes and total health systems performance score. The percentage of successful treatment outcomes is plotted on the Y-axis and the health performance score on the X-axis. From the graph it can be seen that Site 1 had the highest treatment success and highest total health system performance score. Sites 2, 3 and 4 can be seen to have lower health system performance scores and lower rates of treatment success. This graph shows there was an association between successful treatment outcomes and total Health System Performance score (r = 0.99) and that this association was significant (p,0.01). doi:10.1371/journal.pone.0094016.g002 Table 6. Correlation between health system performance and successful treatment outcomes for each domain for MDR-TB patients treated at 4 decentralised sites. to be explored. [40,41] In addition, regular, careful support and supervision improves quality of services significantly. [42] If staff feel supported and confident, they are less likely to seek alternative employment. Moreover, to ensure optimal implementation, district and facility managers need support, supervision and monitoring to ensure they take ownership of a new programme, take responsibility for service re-organisation and re-alignment and provide support. Besides the lack of support for front-line staff at Sites 2, 3 and 4, two routine HR practices contributed to poor HSP. This regular rotation of clinical staff contributed to the loss of valuable skills and experience and instability and a lack of continuity in MDR-TB management. The practice of rotation has to be reconsidered for key clinical positions such as the doctor responsible for MDR-TB services and the nurse-in-charge of the outpatient clinic. If rotation is necessary, a longer rotational cycle would reduce the rapidity with which skills are lost.
Secondly, in resource-constrained settings, a strategy used to reduce personnel costs is to appoint staff as 'acting' managers. Site 4 had an acting facility manager for much of the study period, which, together with a number of other factors contributed to the poor HSP at this site. This stalling tactic for key delivery positions is counter-productive resulting in unmotivated and unproductive staff, poor service implementation and poor patient care.
The importance of the head of the outpatient clinic in health services for chronic and long-term conditions is seldom recognised. In decentralised MDR-TB services, this person is the interface between MDR-TB and PHC services and the community as well as being the gateway to tertiary care. First, she has to ensure that the service is patient-focussed, the patient is supported and constant education is provided to promote adherence. Second, she is responsible for co-ordination of two different programmes (TB and HIV), and for the organisation of different and diverse support service components to ensure health service performance is optimal. And third, she acts as a liaison between the different levels of care so that the patient always receives appropriate treatment.

Limitations
This study was an evaluation of an intervention implemented in the public sector, providing evidence under routine conditions which supports wider applicability of results. However, it was subject to challenges experienced in this sector. Our data used for the evaluation was the data routinely collected by health workers, which at times, was incomplete and inaccurate. Although we collected individual patient data, the routinely collected health system data were not available at an individual level. And, therefore, we were unable to determine the impact of HSP on individual patient TOs. By focusing on health system performance other factors, such as differences in baseline characteristics, which may have influenced TOs were not included in the analysis. Furthermore, the small sample size (4 sites) lessened the power of tests, reduced precision, increased the effect of variability and precluded the use of more sophisticated methods to determine the association of different domains to one another.

Conclusions
This study is a first step toward predicting which health system factors affect treatment outcomes (TOs). In spite of the above limitations, we have shown that there is a trend between HSP and TO and a larger case control or comparative study is warranted. To conclude, we suggest that decentralised MDR-TB management can improve treatment outcomes if district leadership is effective, management takes ownership of the problem and provides support by re-organising and re-aligning health service components, allocating sufficient financial resources, and providing regular visits and assistance in resolving emerging problems. Moreover, a vertical programme can strengthen district level health systems if it is integrated into existing services.
As MDR-TB prevalence increases, health services expand, and different models of care are introduced, we recommend regular monitoring and support of district and facility managers and individual health workers to encourage service integration, guideline adherence and optimize TOs.
In addition, we have identified HR practices that are detrimental to HSP: rotation of staff in key clinical positions and the appointment of managerial staff in an 'acting' capacity. We recommend alternatives to these practices. Table S1.