Data Analysis

Table 1 summarizes the study questions and the sources of data and types of variables used to address each of the study questions. It should be noted that for the first transition tranche in 2009 no survey data were available. Accordingly for that particular tranche, our analysis draws only upon the case studies conducted.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Data sources and type of analysis by Study Question.

https://doi.org/10.1371/journal.pone.0136177.t001

For the transition readiness and institutionalization surveys, responses were translated into English where necessary, and entered into and analyzed in Excel. For measures of alignment three simple response categories were created reflecting full alignment, partial alignment, and no change from Avahan modalities. Analysis of both surveys sought to compare the degree of transition readiness, by round, by state and by population served.

For the case studies, interviews and focus group discussions, data collection was conducted in the local language, recorded, then transcribed and translated into English in India. Data were coded in Atlas.ti using a framework approach and analysis was confirmed between researchers in the U.S. and India. For each case study, a separate report was drafted, synthesizing findings from the data available.

Data were triangulated within and across study components and early results were used to inform later data collection. Results from the first round of data collection were shared and discussed with Avahan and local counterparts. It should be noted that one component of the Avahan transition strategy (namely support to government capacity at national and state level) was not addressed in this evaluation.

Ethics Statement

The study was submitted to and exempted by the Internal Review Board of the Johns Hopkins Bloomberg School of Public Health. It was approved by the YR Gaitonde (YRG) Medical, Educational and Research Foundation, Centre for AIDS Research and Education Institutional Review Board in India. Respondents to all components of the study were asked for and provided written consent, with the exception of community participants in focus group discussions in the longitudinal case studies who, for reasons of limited literacy, provided oral consent that was recorded by the enumerator. These consent procedures were approved by the YRG Institutional Review Board.

What was the transition strategy and how was it implemented?

There was very early recognition of the need to transition the Avahan program to local counterparts. In 2005, barely two years after its initiation, the Avahan technical panel and partners discussed the need to move Avahan towards a “truly sustainable” program. At the time this was considered mainly in terms of strengthening community owners and transferring funding and management responsibilities to government. This commitment to transition was formalized in a 2006 Memorandum of Cooperation with government[18]. A 2008 Avahan strategy document made a first attempt to lay out the transition strategy. This was formalized in a 2009 Memorandum of Cooperation that set out a clear transition timeline and transition responsibilities for government and Avahan[19].

The transition strategy continuously evolved responding to new learning as well as shifts in context, but key elements of the strategy included:

1. Strengthening government systems at national and state level to manage TIs, through the provision of assistance to the Technical Support Unit at NACO, and through funding of state-level Technical Support Units.
2. Building NGO/CBO capacity to run TI programs independently through training for NGO/CBO staff.
3. Enhancing key populations’ capacity to demand and access services and information, and promote their engagement in building an enabling policy and program environment through community mobilization activities conducted by State Lead Partners at TI Program staff.
4. Aligning clinical and non-clinical aspects of the Avahan program with government systems prior to handover. In terms of clinical alignment this involved new clinical protocols (such as for STI syndromic management) and referring clients to government health services such as Integrated Counseling and Treatment Centers (ICTC). Non-clinical dimensions included alignment with NACO norms for budgets, staffing profiles and procurement systems, as well as adoption of the government’s health management information system.
5. Promoting continued government commitment to targeted prevention programs and transition of the Avahan program, through engagement with government in the development of key policies such as the National AIDS Control Program III.

The 2009 transition round was problematic. While upper and middle management levels at NGOs/CBOs running the TI programs and state-level organizations were relatively well informed about transition, frequently peer educators were less well informed, notified just prior to the transition or were unaware that transition was taking place. Transition preparation focused on budget and staffing alignment, but failed to anticipate operational challenges TIs would face post-transition, notably shortages of cash and commodities due to delays in initiating budget disbursements and supplies via government systems. While transition was meant to occur in April 2009, frequently government contracts to NGOs/CBOs were not issued until months later. There was no planning for the kind of post–transition support that SLPs might provide after transition, which sometimes resulted in confusion at the TI level with both SACS and SLPs intervening. In one state the SLP was asked by SACS to cease its involvement with TI programs that had transitioned, thus undermining any post-transition support.

Informants suggested that one reason for the challenges encountered in the 2009 transition round were the short time frame for planning transition. In our case studies we found efforts to prepare transitioning TI programs only started 1–3 months before transition. In addition a lack of experience in collaboration between Avahan and government sometimes manifested itself in mistrust.

By comparison, the 2011 transition round appeared better organized. Relationships between Avahan and government were stronger and communications between these actors were more effective and built on trust. New posts for “transition managers” were established within SLPs, as well as at the central level in BMGF, ensuring a clearly identified point-person with responsibility for the transition. Clear indicators to judge the level of program interventions, and budget lines to support transition were established. There was a longer period of transition preparation, with TIs starting preparation on average 6 months before transition according to the transition readiness survey. In two states, Karnataka and Maharashtra, buffer stocks of condoms, medicines and occasionally funds were built up to address potential gaps in supplies. Further, arrangements for post-transition support, which allowed SLPs to continue to work with TI programs, particularly on community mobilization activities were formally agreed in advance. These improvements were facilitated by the fact that the second transition round came a full two years after the initial round, was relatively small (comprising just 28 TIs), and therefore relatively straightforward to implement.

The 2012 transition round appeared to go smoothly despite its large size. This success built on even earlier preparation (starting on average 9 months before transition), ongoing strong collaborations and effective communication between SLPs, SACs and TSUs. Post-transition support packages, especially for community mobilization, were standardized across TIs. Buffers of funding and commodities were established for all TIs. Overall planning and implementation of transition strategies were guided by a Common Minimum Program for transition. Data from the surveys reinforce the positive impression that case study data provide of effective implementation of transition strategies in later rounds: while 79% of TI managers in the 2011 round agreed or strongly agreed that “the transition experience went smoothly”, 93% of TI managers perceived the last round of transition to have gone smoothly.

How well prepared was the Avahan program for transition?

In terms of developing capacity and promoting engagement of TI staff and key populations, there was a gradual increase over time in the amount of staff training provided to help prepare for transition. For example, overall more field staff attended transition trainings in 2012 versus 2011 (60% of TIs vs. 15%, respectively), and more TI managers received training on changes to TI guidelines in 2012 compared 2011 (83% vs. 67%, respectively). However higher-level staff received more training than lower-level staff, and for the April 2012 transition most staff training for transition took place very late in the process.

In terms of alignment of clinical aspects of the program, Avahan TIs had previously delivered most clinical services for general health and sexually transmitted infections through TI-run centers or alternatively had contracts with private clinics for services. During transition, the delivery of clinical services shifted away from TIs to government facilities. Typically this also implied a shift from broad services for sexually transmitted infections, and indeed general health services, toward a narrower focus on integrated counseling and testing for HIV and syndromic treatment for STIs.

Overall, we found high levels of clinical alignment and very high referral rates to government ICTCs (Table 2). In a number of states, as part of the transition process, government sought to sensitize employees at government clinics so as to improve stigma and discrimination previously felt by clients at these facilities. These efforts appeared successful for FSW, but less so for MSM and transgendered populations, and also varied from one state to another.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Measures of clinical and non-clinical alignment of Avahan programs with government prior to transition.

https://doi.org/10.1371/journal.pone.0136177.t002

Avahan also needed to align managerial and operational aspects of the program prior to transition. We found high alignment of budgets, TI team structure, and reporting and procurement systems (Table 2). Where procurement systems were not aligned, respondents explained that they had buffer stocks, and therefore had not started using government systems. Many TIs described cutting salaries in order to align with government norms, and some also cut staff—typically a difficult process. Often, alignment implied budget cuts for trainings, meetings, travel allowance, outreach, and office expenditure. Salaries for TI workers also declined, although there were a few exceptional circumstances where salaries increased or were protected. Sometimes SLPs managed to negotiate some flexibility around government norms from SACS so as to help protect health workers’ pay. New (higher level) literacy requirements also resulted in difficulties for staff recruitment and led to turnover, especially among key population members who had been recruited as NGO staff.

With respect to government commitment, Avahan was fortunate to be transitioning its program into the context of a highly committed and matured government program, whose policy goals aligned with Avahan HIV prevention objectives. The Third National AIDS Control Program (NACP III)[16] committed to quadrupling government funding for HIV/AIDS and set out a clearer focus than previously on HRGs. While Avahan was too newly established to influence NACP III policy formulation, it was substantially involved in the development of norms and guidelines for NACP III implementation[15]. Avahan staff invested significant energy, with considerable success, in working closely with NACO to develop norms and guidelines that built on Avahan learning. The government delivered on the substantially scaled up funding that was promised, providing adequate funding throughout the course of the program[20].

How well institutionalized is the Avahan program?

A number of key practices that Avahan pursued were identified as critical to the success of the program[14]. These key practices can be grouped into four categories:-

• Management practices, notably extensive use of data at different levels of the system, strong onsite supervision, use of clinical and operational guidelines, a “common minimum package”, and flexible management;
• Funding and supplies, including on-time, adequate and uninterrupted flow of funds and commodities to TIs;
• Peer-educator (PE) support, comprising rigorous performance-based management of PEs, use of pictorial micro-planning tool, and needs based training for PEs.
• Community engagement, including program oversight by committees made up of community members, strong support to community groups, and community-led crisis-response mechanisms to respond to violence or harassment of community members.

The institutionalization survey assessed the persistence of these practices post-transition.

Overall, by the time of the institutionalization survey, 9–12 months after transition, there had been significant program changes across these dimensions. 75% of survey respondents from the 2011 transition round and 69% from the 2012 round agreed with the statement that “The overall program has changed significantly as compared to pre-transition.”

Over 50% of TI program managers noted changes in key management practices (Table 3). In general, these changes in management practices were viewed positively. For example, respondents noted improvements in supervisory structures, clarity of guidelines and use of data for planning and tracking progress. The one exception was in terms of flexibility in management. Across both rounds, 57% of TI program managers viewed the government system to be less flexible than Avahan, and were particularly concerned about the implications of limited financial flexibility such as ability to move money between budget lines or carry funds from one period to the next.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Change in Avahan practices post transition.

https://doi.org/10.1371/journal.pone.0136177.t003

With respect to funding and supplies, close to 70% of TI programs always had sufficient stocks of condoms and medicines, but the remaining 30% had experienced commodity stock outs within a few months post-transition largely due to changes in the supply source and schedules. In all, 10% of TIs regularly had problems with cash flows and less than half of the TIs never had cash flow problems. Further, the transition reduced budgets for 63% (across both rounds) of TIs, which in turn decreased TI staff salaries, travel allowance, and funding for TI events and activities.

In 59% of TIs (across both rounds), performance monitoring of peer educators was felt to have changed, largely for the better. There appeared to be closer monitoring of peer educators, with more concrete performance standards, and termination of contracts for those who did not meet standards. Further, 59% of TIs (across both rounds) also thought that peer educator training had changed, though views were mixed on whether this had led to improvement. Due to more restricted budgets, the frequency of training had declined, but training opportunities may have been better targeted than previously. The specific use of a pictorial microplanning tool to map HRGs had also changed for 43% of TIs, which was largely perceived as positive.

Based upon the institutionalization survey, about one third of TIs experienced changes in community engagement across the three activities considered. The majority of TI program managers viewed these to be changes for the better; however, a different perspective emerged from community members who participated in the focus group discussions conducted as part of the longitudinal case studies. Community members perceived that TI programs had shifted away from providing incentives and community recognition activities towards a more narrowly clinical focus. Budget cuts that removed small amounts of funding for tea and refreshments for community members had disproportionately large effects on the motivation and satisfaction of community members. Shifts in the source of clinical services also resulted in mixed feelings: while many community members appreciated the possibility of being mainstreamed into government clinics, and recognized improvements in government provider sensitivity to their needs, they also mourned the loss of general health services at the TI. Many resented the focus on testing to the exclusion of other activities. MSM and transgender communities appeared particularly adversely affected by the shift.

Were program outcomes sustained after transition?

Figs 3 and 4 show trends in services before and after transition in a pooled sample of TIs that transitioned in 2011 and 2012. Service coverage is captured by the average percentage of HRGs contacted by peer educators per month (Fig 3) and the average number of condoms distributed per HRG per month (Fig 4). Data were standardized around the same transition date, and pooled as there was no clear difference in trends in services between TI programs transitioning in 2011 and 2012.Overall, it appears that service coverage was sustained at least up to six months post transition.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Sustained outcomes: High risk groups contacted by peer educators.

https://doi.org/10.1371/journal.pone.0136177.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Sustained outcomes: Condom distribution to members of high risk groups.

https://doi.org/10.1371/journal.pone.0136177.g004

Elements important for an effective transition

Many of the lessons from this evaluation echo those of the program implementers[9]. We identify five elements of the transition design and implementation that contributed to a successful transition.

1. Extended timeline and phased transition—The extended (2009–2013) timeline allowed for substantial transition preparation. Further, the phased nature of the transition facilitated learning and feedback based on prior transition rounds. We observed increasing investment in and a more standardized approach to transition preparation, as both Avahan and government actors learned what was necessary to support transitioning TI programs and acted upon this learning in later transition rounds. The extended timeline also supported the development of trust between stakeholders and open lines of communication.
2. Substantial investment in transition preparation–Avahan made a major investment in preparing the program for transition, particularly focused on aligning the program, ensuring clear communications and agreements with local stakeholders, and enhancing capacity of local partners to manage the program. Specific transition management strategies emerged after the 2009 transition round and were systematized in a Common Minimum Package for transition ahead of the 2012 transition round.
3. Commitment to transition–Avahan was fortunate to transition to a government that already had strong policy commitments to HIV/AIDS prevention that aligned with Avahan’s own goals. Moreover, the Government of India was prepared and willing to take over the funding and operation of Avahan services. However, this was not purely luck–Avahan sought to shape the policy environment to ensure strong commitment to prevention among HRGs by actively engaging in the development of NACP III. It also signaled a strong commitment to transition, through the memoranda of cooperation that helped convince all actors of the seriousness of transition plans.
4. Communication and trust–communications between key partners at state level (SLPs, SACS, and TSUs), as well those running TI programs were relatively strong and improved over time. Initially, during the 2009 transition round, state level actors demonstrated mutual suspicion, but the need to work together in the transition process meant that gradually trust between actors grew and highly collaborative working relationships were established. Specific activities to build communication and trust between those transitioning the program and those receiving it may be beneficial.
5. Adaptability and willingness to provide additional support in key areas—the initial Avahan transition strategy did not address community mobilization differently from other program elements. However, it became apparent during the transition process that government was less well placed, both in terms of orientation and expertise, to support community mobilization activities than the Avahan program. Avahan demonstrated flexibility by seeking to address this issue which was not part of the original plan. The post-transition support agreements, which had a strong focus on community mobilization helped mitigate, although perhaps only temporarily, some of the negative consequences of transition for community mobilization.

Limitations

There are several limitations to the evaluation. Taking a broad perspective, this evaluation had an adequacy design[21], and we are therefore unable to causally link the transition preparations made to the smoothness of transition, or sustained program coverage. Instead our findings suggest that a high degree of transition preparedness contributes to smooth transition and the sustainability of program coverage.

There were a number of specific design weaknesses in the evaluation. First, while we assessed the amount of training provided, trainings do not necessarily translate into higher capacity, and we have no data on the impact of training on skills. Second, there was limited buy-in to the evaluation from the Government of India and this restricted opportunities for checking interpretation of data with government stakeholders and made it difficult to secure national or state-level data on TI service coverage post-transition. Thus, we were not able to compare the data collected from the transitioning Avahan TI programs with the broader population of TIs. Third, the evaluation did not measure the extent to which key Avahan practices were conducted prior to transition and instead relied on TI manager recall of how things had changed. Fourth, this evaluation collected data from TIs immediately before transition and approximately one year post-transition; however, many program changes were still transpiring at this time, and the Indian health system is also changing rapidly, thus the true long run effects of transition may still be emerging.

Finally, some features of the Avahan context including the high-level government policy commitment to HIV/AIDS prevention, the availability of significant government funding to support this commitment and relatively high government capacity (despite frequent challenges of high staff turnover) may limit transferability to other contexts.