The Avahan Transition: Effects of Transition Readiness on Program Institutionalization and Sustained Outcomes

Background With declines in development assistance for health and growing interest in country ownership, donors are increasingly faced with the task of transitioning health programs to local actors towards a path to sustainability. Yet there is little available guidance on how to measure and evaluate the success of a transition and its subsequent effects. This study assesses the transition of the Avahan HIV/AIDS prevention program in India to investigate how preparations for transition affected continuation of program activities post-transition. Methods Two rounds of two surveys were conducted and supplemented by data from government and Avahan Computerized Management Information Systems (CMIS). Exploratory factor analysis was used to develop two measures: 1) transition readiness pre-transition, and 2) institutionalization (i.e. integration of initial program systems into organizational procedures and behaviors) post-transition. A fixed effects model was built to examine changes in key program delivery outcomes over time. An ordinary least square regression was used to assess the relationship between transition readiness and sustainability of service outcomes both directly, and indirectly through institutionalization. Results Transition readiness data revealed 3 factors (capacity, alignment and communication), on a 15-item scale with adequate internal consistency (alpha 0.73). Institutionalization was modeled as a unidimensional construct, and a 12-item scale demonstrated moderate internal consistency (alpha 0.60). Coverage of key populations and condom distribution were sustained compared to pre-transition levels (p<0.01). Transition readiness, but not institutionalization, predicted sustained outcomes post-transition. Transition readiness did not necessarily lead to institutionalization of key program elements one year after transition. Conclusion Greater preparedness prior to transition is important to achieve better service delivery outcomes post-transition. This paper illustrates a methodology to measure transition readiness pre-transition to identify less ready organizations or program components in advance, improving the likelihood of service sustainability. Further research is needed around the conceptualization and development of measures of institutionalization and its effects on long-term program sustainability.


Introduction
One of the greatest challenges of donor-led programs is to sustain activities and outcomes beyond the project funding period. Too often projects end abruptly when donor funding discontinues. This is particularly the case when the donor has supported service delivery through systems that are parallel to and separate from government's own health systems. Sometimes donors invest in planning for a transition process whereby key elements of the program are handed over to local partners including government, but this kind of planned transition process is relatively rare [1]. Quantitative studies measuring the success of transitions are uncommon.
The challenges of transition are shared across many donor-led programs, and is a particularly pertinent question at this time given tempered growth in development assistance for health [2,3] as well as for HIV/AIDS [4,5], and plans to promote country ownership and transition within the U.S. President's Emergency Plan for AIDS Relief (PEPFAR) [6][7][8]. This challenge is also relevant to ongoing graduation of countries from donor programs such as the Global Fund to Fight AIDS, Tuberculosis and Malaria [9] and Gavi, the Vaccine Alliance [10][11][12].
Transitions, if done well, provide an opportunity for donors to steward successful programs into the capable hands of local actors and integrate them into domestic country health systems. A successful transition involves formally handing over a program to one or more local partners and ensuring that the key outcomes of the program are sustained over time. An effective transition is often characterized by alignment and smooth transfer of key services, good communication about transition preparations and plans, and sufficient capacity among local program owners [1]. Preparations pre-transition are frequently described to be essential in this process [13,14].
Transition may also involve a process of change, as donors and local actors may have different structures, management practices and organizational cultures. The extent of continuity may be captured by institutionalization, which considers the extent to which the initial program systems are integrated into organizational procedures and behaviors post-transition [15]. Institutionalization is seen to be important in program sustainability, considering the extent to which innovation and learning not only gets adopted and continued, but also gets reflected in institutional standards and norms that govern multiple organizations within the broader health system [15]. To date, there has been a dearth of empirical work to measure and assess the degrees of institutionalization post-transition [16].
Increasing attention has been given in recent years to examine whether various outcomes of health programs are sustained [17,18]. A review of empirical literature identified 125 studies on sustainability which examined the performance of health programs after initial implementation [19]. Many studies reported that partial sustainability was more common than continuation of the entire program or intervention. However, the wide array of studied outcomes and durations made it difficult for reviewers to generalize the overall extent to which new programs and practices are sustained. The review also found the majority of studies to be retrospective, with few studies employing rigorous methods of evaluation. The large majority of studies took place in high income countries.
This study uses data from an assessment of the transition of the Avahan India HIV/AIDS prevention initiative to investigate how activities undertaken to prepare Avahan for transition affected continuation of program activities post-transition. Prior to transition, Avahan was fully supported by the Bill and Melinda Gates Foundation, whose staff managed grants or contracts with a range of international and national Non-Government Organizations (NGOs) [20]. These organizations in turn contracted with smaller NGOs and Community Based Organizations (CBO) to provide HIV/AIDS prevention services for key populations known as Targeted Interventions (TIs) [21,22]. Key populations in this study included female sex workers and/or high-risk men having sex with men. Transition experiences from the community perspective were examined separately through qualitative studies [23].
Service delivery of Avahan was transitioned to the Government of India in three increasingly larger rounds of transition in 2009, 2011 and 2012. This study examines the transitions that took place in tandem in southern Indian states. Post-transition, the government took on responsibilities to fund and manage TIs, where they must adhere to national guidelines set by the National AIDS Control Organization (NACO), while being accountable to State AIDS Control Societies (SACS). TIs were transitioned to four different SACS where application of NACO norms varied by state in practice, making them a heterogeneous group. While the majority of TIs transitioned in original forms, some TIs had split because they were too large for NACO guidelines, or separated by key populations. Other TIs merged because they were too small for NACO norms, or discontinued. Other papers provide a fuller overview of the Avahan transition process, and the strengths and weaknesses of this process [1,[24][25][26][27].
This study examines the transition by focusing on three core elements: 1) how well prepared programs were prior to transition, 2) whether key elements of the program were institutionalized post-transition, and 3) whether outcomes were sustained through the transition. In particular, we examined relationships between transition readiness, institutionalization and outcomes to provide insights in the effectiveness of transition planning. We hypothesized that transition preparedness may explain what happens post-transition in terms of (i) institutionalization of key program elements one year after transition and (ii) sustained program delivery. This paper is unique in developing measures of transition preparedness and institutionalization, using data from both before and after transition to analyze the relationship between these variables, and examining linkages with sustainability of service outcomes.

Materials and Methods
Surveys were carried out in five southern states of India where Avahan was transitioned to local partners: Andhra Pradesh (now Andhra Pradesh and Telangana), Karnataka, Maharashtra, and Tamil Nadu. We conducted two rounds each of the transition readiness and institutionalization surveys. The transition readiness survey was conducted at the time of transition, whereas the institutionalization survey was carried out 12 to 18 months post-transition. All of the TIs from the 2011 round of transition were surveyed, whereas we used a list of transitioning TIs stratified by state to randomly select TIs from the larger 2012 round of transition. Data from 2011 and 2012 rounds of transition were combined together in the analysis to achieve sufficient sample size.
The transition readiness survey examined a range of indicators that sought to assess how well prepared the TIs were for transition. Through a literature review and conceptual framework of a transition logic model which was developed to evaluate the Avahan transition [1], we identified three elements of transition readiness: 1) capacity, 2) alignment, and 3) communication. Capacity indicators captured key operations of the TI, such as linkages made with government health facilities, formation of key population community groups and functioning of crisis response committees [28]. Alignment indicators measured levels of preparations made by the TI towards meeting NACO norms in areas such as team structure, budgeting and reporting. Communication captured whether staff were informed about the transition, transition plans incorporated staff inputs and project coordinators received training for the transition. Indicators were considered important where there were government norms to assess the extent to which TIs are aligned, as well as questions to understand how well prepared the TIs were for the transition in terms of capacity and communication [24,29]. In addition to interview questions, the survey also included a review of data in the Avahan Computerized Management Information System (CMIS) from which we abstracted relevant indicators. For each indicator gathered from interviews and documents, we defined 3 levels (0 = low, 1 = medium and 2 = high) of transition readiness based on how well prepared TIs were for transition. For example, we classified the extent to which TIs met NACO norms, established linkages with government health services or had informed staff about the transition. The transition readiness survey captured 21 indicators, including 18 which were asked to TI managers and 3 which were abstracted from CMIS data.
The institutionalization survey examined whether transitioned TIs had institutionalized characteristics of the original program post-transition. Specifically, we asked whether transitioned TIs were continuing to regularly practice certain characteristics of the Avahan program. Core Avahan characteristics were identified from an earlier Delphi study, of which 13 of the 17 features were considered to be relevant post-transition [30]. The institutionalization survey included 18 interview questions which asked how frequently TIs carry out identified Avahan practices (i.e. regularly, sometimes, never). We then asked about Avahan practices in 20 statements using a five-point Likert scale, including whether the transition experience went smoothly overall. Additional 20 questions asked whether these practices changed due to the transition, which institution (i.e. government, TI or others) brought about the changes and whether these changes were for the better, worse, or made no difference. Interview questions were supplemented by a review of the government's CMIS for relevant indicators.
Program outcome data were gathered from our surveys, the Avahan CMIS database and TI proposal documents. To observe changes pre-and post-transition, we selected indicators which were recorded in both the Avahan CMIS (for the pre-transition period) and the government's CMIS (for the post-transition period). Available outcome indicators were limited based on the data that TIs gathered and reported. As a primary outcome indicator, we used the average percentage of key populations contacted by peer educators per month. Specifically, Avahan CMIS and government CMIS had collected the number of female sex workers and high-risk men having sex with men who were newly or repeatedly contacted by peer educators using counseling materials. These contacts were divided by the number of key populations served by each TI. As a secondary outcome indicator, we examined the average number of condoms distributed per key population per month. Sizes of key populations for each TI were obtained from TI proposal documents or the Avahan CMIS database [31]. Pre-transition data were abstracted from the Avahan CMIS database for 18 TIs to supplement the government CMIS data received from TIs. Panel analysis was used to plot observations. Heteroskedasticity was assessed using the Breusch-Pagan / Cook-Weisberg test [32,33]. For each outcome a fixed effects model was developed with robust standard errors.
Measures of transition readiness and institutionalization were separately constructed through exploratory factor analysis. Based on the literature, we hypothesized the transition readiness measure to be multi-dimensional capturing elements of capacity, alignment and communication, and developed 3 sub-scales for this measure. However, there was no prior literature to inform the items to construct the scales. The literature also did not provide evidence to suggest the dimensionality of the institutionalization measure. Items for both scales were factor analyzed using iterated principal factor extraction with Varimax orthogonal rotation. The number of factors was determined using a scree test with eigenvalue greater than one. Items were analyzed and retained based on factor loadings, item-to-total correlation and inclusion of diverse elements of the concepts. Construct validity was examined by correlation analyses against other theoretically related constructs. Internal consistency was assessed by Cronbach's alpha.
Using the two developed measures, ordinary least squares regression was subsequently carried out to examine the relationship between transition readiness and program outcomes directly, and indirectly through institutionalization (Fig 1). Relationships between transition readiness subscales and outcomes were also separately examined. For both outcome indicators, we examined the relationship of average outcomes 12 months pre-transition and 6 months post-transition. Univariate analyses were followed by multivariate analyses controlling for rounds of transition, whether the TI had split at the time of transition, NGO or CBO status of the TI, target key population (female sex workers, high-risk men having sex with men, or both), and state.
This study was ethically reviewed by Johns Hopkins School of Public Health Institutional Review Board (IRB No. 3157) and exempted as the study collected data from key informants but not data about individuals. In India, it was reviewed and approved by YRG Care Institutional Review Board (IRB No. 1423). Participants provided written informed consent to participate in the study.
Overall, many TIs scored high across transition readiness indicators, demonstrating various adjustments made to meet NACO norms, build capacity and communicate prior to the handover. Most TIs had appropriately communicated and aligned the reporting systems (average score of 1.88 out of 2), staff structure (1.95 out of 2) and budgets (1.88 out of 2) prior to transition. Where transition readiness scores were lower, we found that some TIs delayed alignment on procurement of medicines due to buffer stocks (0.96 out of 2), or had removed condom outlets in hotspots because of vandalism and privacy concerns (1.21 out of 2). Scores were also lower for visits to Integrated Counselling and Testing Centers (ICTC) post-referral (1.10 out of 2), where government CMIS records suggested that 80% of referrals were actually seen at ICTC centers compared to 100% requested by NACO (Table 2).
Based on an item analysis of 21 transition readiness items, those with negative factor loadings and item-to-total correlations below 0.2 were removed. This resulted in 15 items in the transition readiness scale explaining 81% of the variance. Items were summed unweighted, with a mean score of 24.34 (standard deviation 3.09) and a range of 16-29. Indexed to a 0-100 scale, the mean was 81.11 (standard deviation 10.30). The scale demonstrated acceptable internal consistency with a Cronbach's alpha of 0.73. Some evidence toward construct validity was found where the transition readiness scale was correlated with TIs reporting smooth transition experience at r = 0.29 (p = 0.03).
Exploratory factor analysis revealed 3 factors which appeared to reflect the three elements of transition readiness from the literature: capacity (eigenvalue 3.92), alignment (eigenvalue 2.57), and communication (eigenvalue 2.08), although not all items were aligned with the factor with which we had originally associated them. Three subscales were then developed for sub-analyses   Table 1). Frequency of practice of Avahan characteristics were examined on a scale from 0 to 2 (0 = never, 1 = sometimes or 2 = regularly). Most TIs reported actively using data for program planning (average score of 1.99 out of 2), regularly using the pictorial micro-planning tool (1.96 out of 2), and practicing rigorous performance monitoring of outreach workers (1.99 out of 2) [see Table 4]. However, some Avahan characteristics were less regularly maintained posttransition, such as providing flexibility on budgets (0.41 out of 2) and allowing exceptions to operating norms (0.89 out of 2). Another Avahan characteristic to have on-time, adequate and uninterrupted flow of funds to the grassroots level was often not continued, where 7 TIs (10%) reported regularly and 31 TIs (44%) sometimes facing challenges with cash flow that affected their operations during the 12 months post-transition.
Factor analysis revealed one dominant factor with an eigenvalue of 2.35. Among the 16 original items, 4 were removed due to negative factor loadings and low item-to-total correlation below 0.25 (Table 4). Dropped items included those with high average values with little variation, where items were less useful to differentiate TIs in a scale. The institutionalization scale was then developed using 12 items, with an average score of 18.04 and standard deviation of 2.59. On a scale from 0-100, this corresponds to a mean of 75.17 and standard deviation of 10.79. The scale explained 52% of the variance and demonstrated moderate internal consistency with a Cronbach's alpha of 0.60. In assessing construct validity, the institutionalization scale was appropriately but weakly negatively correlated with TIs reporting that the overall program has changed significantly as compared to pre-transition r = -0.17 (p = 0.15).

Sustained Outcomes
TIs with outcomes data both pre-and post-transition were limited even after combining multiple data sources (government CMIS data obtained from TIs, Avahan CMIS database and TI proposal documents). Data were available for 55 TIs (79% of the institutionalization sample) for the primary outcome indicator: average percentage of key populations contacted by peer educators per month. Data were obtained for 64 TIs (91% of institutionalization sample) for the secondary outcome indicator: average number of condoms distributed per key populations per month. Both outcomes were relatively stable between 12 months pre-transition and 6 months posttransition. While there was a minor drop in the average percentage of key populations contacted by peer educators one month post-transition, there were no trends observed comparing pre-and post-transition in the levels of either indicator based on the fixed effects model (Table 5). For both outcomes, the data was found to be heteroskedastic, where variance of the data reduced post-transition (p<0.001).

Effect of transition readiness and institutionalization on sustained outcomes
Based on a regression analysis, we observed a statistically significant relationship between transition readiness and sustained outcomes post-transition, for both key population coverage (2.47, p<0.01) and condom distribution (2.17, p = 0.03). The transition readiness measure was also predictive of outcomes comparing 6 months pre-and post-transition, for key population coverage (0.03, p<0.01) and condom distribution (0.04, p = 0.02). Examining the subcomponents of the transition readiness measure, this relationship was strongest across the dimensions of alignment and communication ( Table 6). None of the variables we controlled for were statistically significant, including rounds of transition, NGO or CBO status of the TI, target key population or state. However, this relationship between transition readiness and service delivery outcomes did not hold through institutionalization. Specifically, the transition readiness measure was weakly associated with institutionalization (0.01, p = 0.91) and significant associations were not observed between institutionalization and service outcomes (3.06, p = 0.08 for key population coverage; 0.25, p = 0.85 for condom distribution). While transition readiness was predictive of sustained outcomes, there were no indirect effects observed through institutionalization.

Discussion
We find that transition readiness can explain sustained program delivery in an HIV/AIDS prevention program in India handed over from a donor to the local government. Specifically, TIs with greater preparedness prior to the transition were more likely to achieve better service delivery outcomes post-transition, and this was also true for dimensions of the transition readiness scale, including alignment and communication. Although the four distinct SACS implemented NACO norms with some variation, we found that the relationship between transition readiness and sustained outcomes did not significantly differ by state.
Yet we also found that transition readiness did not necessarily lead to institutionalization of key program elements one year after transition, nor was institutionalization predictive of sustained program delivery. These results may be explained partly by difficulties in measuring the construct of institutionalization. While we developed our institutionalization measure based on the core characteristics of the Avahan program identified through a Delphi study [30], the measure may not have sufficiently captured the construct of institutionalization as indicated by the scale's moderate internal consistency and modest proportion of explained variance. In addition, our study design did not capture the extent of Avahan practices prior to transition, thus making it difficult to assess whether there was truly a decline in the prevalence of these practices.
Furthermore, there appears to be some conceptual misalignment between measures of alignment with government norms in the transition readiness scale vis-à-vis the continuation of specific Avahan program characteristics post-transition captured through institutionalization. For example, ensuring on-time, adequate and uninterrupted flow of commodities was viewed to be an Avahan characteristic, yet TIs had to change their procurement with government supply systems to be transition ready, which likely led to interruptions in commodities supplied. Conversely, TIs that were better prepared for transition, as embodied by wellinformed and skilled staff and high institutional capacity, may have been more inclined to innovate and change as compared to TIs with less capacity. Indeed, there appeared to have been substantial change in the TIs surveyed since transition, and much of this change was positively perceived by TI managers. As programs evolve over time, the linkages between transition readiness, institutionalization and sustained impact become more complex and would benefit from dynamic systems modeling [34].
While there was some evidence to suggest that the three elements of transition readinesscapacity, alignment and communication-based on a literature review and conceptual framework [1] was supported by the data, we found it to be far from conclusive. For instance, the loadings of the three factors did not align entirely with our initial conception of the three subscales. In addition, raising the item-to-total correlation cutoff level in the analysis removed variables and collapsed the subscales into one factor. This provides good fodder for future research to look into the sub-components of the transition readiness measure.
Our main findings support the largely grey literature in this area describing transitions and sustainability [18,19,[35][36][37]. It also adds to the limited measurement literature around transitions and institutionalization [38][39][40][41]. While Goodman et al provides an eight-factor model on institutionalization of health promotion programs, we found it difficult to apply this in our study in the context of an evolving and transitioning project with a short follow-up time period [40]. The mostly descriptive accounts of how communication and transition planning affect the overall success of transition of HIV/AIDS programs [13,14,42] are supported through our analysis.
Some additional study limitations are important to note. First, the study was limited by the small sample size of TIs that took part in the program transition. While we surveyed 100% of TIs that transitioned in round 1 and 34% from round 2, the sample size of 80 TIs limit the power in our statistical analyses. Moreover, some TIs were split or discontinued, making it difficult to track their status post-transition. The quality of administrative data gathered through CMIS was also a limiting factor, where some data points were missing and restricted possible service delivery outcomes for analysis. Where possible, we supplemented the government CMIS data by extracting data from other sources such as the Avahan CMIS database and TI protocol documents. Finally, our analysis examined institutionalization one-year post transition, which may not have provided sufficient time for program characteristics to be reflected in institutional standards and norms, or capture the types of feed-back loops in the diffusion of service innovations [16]. Despite these limitations, this paper adds value by evaluating the transition of a large-scale donor-led program to local counterparts both prior to and after the transition.
This study contributes to the literature by developing approaches to measure transition readiness and institutionalization in the context of a health program transitioning from a donor to local partners. While our measure particularly of institutionalization appears imperfect, this study suggests that it may be practical and useful to assess transition readiness quantitatively prior to transition. While such measures should be tailored to fit the nature of the program being transitioned, it appears that they may be a good indicator of service sustainability post-transition, as well as helping to identify needs for further transition preparation.

Conclusion
This study offers important lessons for future transitions of donor programs. Specifically, we illustrate a methodology to measure transition readiness prior to transition, which could be predictive of service delivery outcomes post-transition. Such a measure could identify less ready organizations or program components in advance, improving the likelihood of service sustainability. Further research is needed around the conceptualization and development of measures of institutionalization and its effects on long-term program sustainability.