There is a critical need to increase availability of evidence-based practices in community mental health systems (Beidas and Powell 2016; Powell et al. 2015). Significant progress has been made in defining criteria for high-quality evidence (e.g., systematically collected data, methods that establish causality, replicable findings) and identifying practices for which such evidence is available (see e.g., Blueprints for Healthy Youth Development, California Evidence-Based Clearinghouse), yet identification does not guarantee implementation. Among numerous remaining challenges, cost is a critical barrier to implementation of evidence-based practices in these systems. Well-specified and feasible financing strategies are therefore needed to support implementation efforts (Bond et al. 2014; Proctor et al. 2011; Roundfield and Lang 2017).

One innovative implementation financing strategy is pay-for-success (PFS), also known as social impact bonds. Under a PFS contract, private-sector investors fund the implementation of an intervention targeting an important social welfare problem. If the intervention provides value to the public sector, as determined by an independent evaluation, the investors will receive a payout from the government (Segal et al. 2016; Social Finance, Inc. 2012). Since the first PFS contract was launched in 2010, this private-public partnership strategy has been applied to an increasingly diverse array of interventions that promote social welfare, including supportive housing, early childhood education, and nurse home-visiting for pregnant women. The United States currently represents approximately 20% of the estimated 108 PFS contracts globally and 50% of the total funds invested (Iovan et al. 2018; Social Finance, Inc. 2018).

A recent review of PFS contracts concluded that the approach has considerable potential to impact population health, but also emphasized that PFS contracts should finance implementation of interventions with a strong evidence base (Lantz et al. 2016). Interventions without a strong evidence base may not produce interim or longer-term results and there will not be a public sector payout to the investors. To address this and other challenges, Lantz and Iovan proposed seven criteria—listed in Table 1—that can be used to identify interventions that are appropriate for PFS financing (Lantz and Iovan 2017). Of the many possible approaches that could be taken, these criteria were most appropriate for our research given that they provide a generalizable evaluation of a practice’s suitability for PFS, while still acknowledging administrative, stakeholder, and political considerations. They are also consistent with complimentary sets of criteria (e.g., Urban Institute 2016) that are more suitable for subsequent, context-specific assessment of the feasibility of a particular PFS project, using a particular evidence-based practice, in a specific governmental and population context.

Table 1 Criteria for selection of interventions in pay-for-success financing
Full size table

The purpose of this brief is to illustrate the potential of PFS as a financing strategy for evidence-based practices in mental health by applying Lantz and Iovan’s (2017) criteria to a group of related interventions based on the multisystemic therapy (MST) model (Henggeler et al. 2009). MST was initially developed as an intervention for serious antisocial behavior in youth. It has well-established clinical effectiveness and economic benefits in that population (Dopp et al. 2017; McCart and Sheidow 2016) and has been widely disseminated by a purveyor organization, MST Services. Furthermore, the MST model has been adapted to address other complex behavioral problems in youth and families (MST Services 2017), with varied levels of evidence available for each adaptation. However, MST is challenging for state agencies to implement due to its complexity and costs (Dopp et al. 2018a), especially because many costs accrue up front (e.g., site assessments, initial training, administrative changes) before the intervention starts affecting outcomes. Traditional government budgets have often prioritized remediation of the highest-risk individuals in a population (even when it relies on expensive interventions such as secure confinement) over preventive or rehabilitative approaches like MST (National Research Council 2013; see also Iovan et al. 2018). The combination of high initial costs and high potential returns on investment has generated considerable interest in PFS contracts as an alternative funding mechanism for MST (Roberts and Cameron 2014; University of Denver 2018).

We applied the first six PFS intervention selection criteria to the full range of MST adaptations, to illustrate the process of evaluating and comparing among interventions with varied evidence bases. We did not examine Criterion #7 (no significant political or stakeholder challenges) in our analysis because it is context-dependent; however, we discuss later how that criterion could be applied to our findings. We based our conclusions about the remaining six criteria on information from a number of sources, including an annual report by MST Services (2018b) that summarizes all published research on MST; relevant studies that were not included in that report because they were not published [e.g., results from the Washington State Institute of Public Policy (WSIPP 2017) cost-benefit model] or published too late; and lists of licensed teams for MST and its variants (MST Services 2018a). We specifically used the lists of licensed teams to evaluate Criterion #6, i.e., the ability of provider organizations to implement MST and its adaptations without significant administrative challenges. Given that MST licensure involves continuous participation in a comprehensive quality assurance/improvement system (e.g., training, expert consultation, outcome monitoring) designed to promote successful implementation, we considered the presence of licensed teams a key indicator for that criterion.

Finally, when making determinations about whether the PFS intervention criteria were met for a given MST adaptation, we incorporated a recently proposed framework for “scaling out” interventions (Aarons et al. 2017). Drawing on well-established external validity theories (e.g., Campbell 1957) and multilevel mediation modeling, those scholars describe how evidence-based practices can “borrow strength” by considering evidence from previous effectiveness trials alongside new evidence for the practice’s effectiveness with a different population and/or delivery system—but only to the extent that core intervention processes are maintained during scale-out. Consistent with that framework, we considered MST adaptations to borrow moderate strength from standard MST when they involved changes in population (Type I Scale-Out) or delivery system (Type II Scale-Out) only, but borrow minimal strength when they involved changes in both population and delivery system (Type III Scale-Out).

Our conclusions regarding how well MST and its adaptations meet key PFS intervention criteria are described below and outlined in Table 2. We found that standard MST and three of its adaptations [MST for serious conduct problems, problematic sexual behaviors (MST-PSB), and child abuse and neglect (MST-CAN)] have significant potential for PFS financing. Seven additional MST adaptations failed to meet one or more of the criteria considered.

Table 2 Application of pay-for-success intervention selection criteria to multisystemic therapy and 10 of its adaptations
Full size table

1. MST Addresses Problems of Interest to the Public Sector

MST was designed to address serious, complex problems related to youth behavioral health. In recent years, the nascent field of PFS financing has moved beyond its early focus on “cost savings” to increasingly emphasize practices that provide benefits to recipients, taxpayers, and society at large (i.e., high-value practices; Iovan et al. 2018). Thus, investors and government entities that are dedicated to increasing community well-being and addressing widespread societal problems (e.g., criminality, child maltreatment) would likely view MST and its adaptations as targeting problems that are worth paying to address.

2. MST and Some Adaptations have a Strong Evidence Base

MST adaptations vary widely in terms of their stage of development on the continuum from pilot studies to large-scale transport (see MST Services 2017) and, thus, there is also considerable variability in the amount and quality of studies examining each adaptation. Both standard MST and MST for serious conduct problems clearly meet Criterion #2, with a total of 23 trials between them. This represents a considerable body of evidence from which other adaptations can “borrow strength.” MST-PSB, MST-CAN, and MST Health Care all had a much smaller number of published studies, but still met Criterion #2 because those studies had generally favorable results and/or borrowed strength. The remaining MST adaptations either lacked sufficient research evidence or the existing research evidence was not sufficiently favorable.

3. MST and Some Adaptations are Economically Attractive to the Public Sector

Far fewer studies have examined the economic impact (i.e., savings or value produced) of MST and its adaptations, even in cases where the evidence base for clinical outcomes is robust. Standard MST and MST for serious conduct problems again had the most evidence regarding economic impact; eight of these studies showed economic benefits of MST (e.g., returns of up to $5.04 per dollar spent; Dopp et al. 2014) and the remaining two that were unfavorable were limited in scope. In keeping with the emphasis on value over budgetary cost savings, most of the benefits produced by MST accrued through avoided crime victim expenses, particularly intangible benefits (i.e., reduced pain and suffering), though a significant minority of benefits were taxpayer savings in the juvenile/criminal justice and Medicaid behavioral health sectors (see e.g., Dopp et al. 2014; Dopp et al. 2018a). It is also notable that MST generated most of these benefits within 2 years of intervention delivery.

The other adaptations that met this criterion were MST-PSB and MST-CAN, and both of those were based on the results of one or two cost-benefit analyses (with the analysis for MST-CAN having just been published this past year; Dopp et al. 2018b) but again could borrow strength. For the other adaptations with evidence for economic impact, that evidence was either too limited to meet this criterion (i.e., only examined a narrow range of cost offsets) or was difficult to interpret because of limitations in the overall evidence base for clinical effectiveness of that adaptation.

4. Key MST Clinical Outcomes are Clearly Defined and Quantifiable; and 5. Those Outcomes are Attained in a Reasonable Time Period

The research base for MST exhibited a number of strengths with respect to these two criteria. With the exception of MST-Prevention, every MST adaptation had expected clinical outcomes that were: for Criterion #4, well-defined (e.g., recidivism, out-of-home placement); and for Criterion #5, reasonable in their timeframes (i.e., can be achieved within 1–2 years of beginning intervention).

6. MST and Some Adaptations can be Implemented Without Significant Administrative Challenges

Most licensed MST teams are for MST with serious juvenile offenders/conduct problems (MST Services 2018a does not distinguish between these teams), MST-PSB, or MST-CAN. There are also a limited number of licensed teams for MST-Psychiatric and MST-Family Integrated Transitions (MST Services 2018a). Thus, we considered standard MST and those five adaptations capable of being implemented without significant administrative challenges.

In conclusion, we argue that standard MST for serious juvenile offenders and three adaptations (MST for serious conduct problems, MST-PSB, and MST-CAN) meet the six selection criteria we evaluated, and thus are strong candidates for PFS financing, a novel public-private financing approach to social welfare interventions. Our analysis illustrates the potential of PFS financing to help governments implement MST, a complex intervention that has considerable up-front costs yet also provides cost savings and value (i.e. the behavioral or mental health outcomes that are worth the costs) within a reasonable time period.

One limitation of our analysis is that governments will also need to consider Criterion #7, “No significant stakeholder or political opposition,” before investing in the implementation of MST through a PFS contract. It was not possible for us to globally evaluate this criterion, but we suggest that the extensive dissemination of MST thus far—with over 500 licensed teams in 15 countries (MST Services 2018a)—indicates that it can be met across a wide variety of settings and cultural contexts. In addition, more economic evaluations of MST are also needed, particularly those that examine the impact of large-scale implementation and/or MST adaptations. Even the extensive pool of economic studies for standard MST was limited by a preponderance of studies conducted by MST developers and their direct collaborators, except for the WSIPP (2017) evaluations that—while rigorous—had not undergone peer review.

Overall, PFS shows considerable promise as a financing strategy for implementation of a complex, expensive evidence-based practice (i.e., MST) through private-public partnerships. These findings suggest that other intensive services for high-risk populations in mental and behavioral health might be good candidates for PFS contracts, pending review of the relevant evidence. For example, the Nurse–Family Partnership (Karoly et al. 1998) and the Perry Preschool Program (Nores et al. 2005) have produced favorable clinical and economic outcomes, whereas findings on the economic attractiveness of wraparound services (Swenson et al. 2000) and the Fast Track program (Foster et al. 2006) were less convincing. Evaluation using the PFS intervention selection criteria (Lantz and Iovan 2017) would be a useful first step to identify which practices are the best candidates for using PFS financing to fund their implementation.

Finally, it is important to bear in mind that numerous factors beyond intervention selection (e.g., legal and policy challenges in PFS payouts, knowledge and technical skills, level of local support) affect the capacity of a given government to execute the PFS financing model (Lantz et al. 2016; Segal et al. 2016; Social Finance, Inc. 2012). If a particular government were interested in using PFS to implement MST, they would need to determine how various government entities would be involved in each aspect of the contract based on what makes sense for their context. For example, a particular government entity might be designated to provide the PFS payout, though some of the most successful administrative structures for PFS contracts have involved states dedicating centralized funds for these payouts. This helps avoid budgetary conflicts of interest when savings accrue across different government sectors or agencies (Lantz et al. 2016). The promise of PFS as a financing strategy for any given evidence-based practice—MST or otherwise—is dependent on local capacity to execute a PFS contract, which should be established prior to the intervention selection process illustrated in this brief.