A Tentative Theory of Change to Evaluate Jurisdictional Approaches to Reduced Deforestation

Introduction

The focus on tropical forests–and their contribution to climate change mitigation–is increasingly prominent in the international political agenda (Griscom et al., 2017; Seddon et al., 2019) as evidenced by recent commitments to the Sustainable Development Goals (halt deforestation by 2020), Bonn Challenge (restore 150 m Ha. by 2020) and Paris Climate Agreement (keep global temperature increase below 2°C). In the latter, forests and natural solutions are considered to be able to provide up to one third of the mitigation required (Griscom et al., 2017).

Jurisdictional approaches (JAs) to holistically address forest and land use across one or more legally defined, primarily subnational, territories have recently gained global attention (Boyd et al., 2018). JAs have been proposed and implemented in various places because they were supposed to bring effectiveness gains in terms of deforestation reduction relative to approaches that are purely national, project-based or value-chain-based. The origin and enthusiasm for jurisdictional approaches is highlighted in the policy dialogue on Reducing Emissions from Deforestation and Degradation (REDD+) that evolved from pilot project implementation and national level readiness activities to subnational jurisdictional REDD+ programs (Fishman et al., 2017; Boyd et al., 2018; Hovani et al., 2018). JAs have also gained traction as a way to engage private companies committed to achieving zero deforestation supply chains (Wolosin, 2016). JAs origins can also be linked to landscape approaches (Paoli et al., 2016; Boyd et al., 2018) with which they share a number of principles (e.g., multi-stakeholder, multi-sector, multi-objective, landscape-scale) but use a different scope of action. The effectiveness promise of JAs primarily lies in the potential benefits gained from using government structures as a way, for example, to institutionalize change in land use rules, incentives or laws. Besides, the scale of JAs also makes it possible to work with specific, locally relevant causes of deforestation while achieving significant quantity of deforestation reduction and deforestation free commodities.

Implementation of JAs has been underway for over a decade and a recent study identified nearly 40 initiatives worldwide including the States of Acre and Mato Grosso in Brazil and the region of Madre de Dios in Peru (Stickler et al., 2018). However, despite increasing popularity amongst scientists and practitioners, there is a lack of robust empirical data documenting their effectiveness in delivering environmental, social and economic outcomes (Boyd et al., 2018; Hovani et al., 2018). The Nature Conservancy (TNC) concedes that “it is hard to know what jurisdictional programs did, when they did it, whether it had the intended impacts, and whether these results were greater than in comparable areas without jurisdictional programs” (Hovani et al., 2018). Emerging evaluations of jurisdictional REDD+, although not measuring their impact per se, suggest that they might fall short in delivering intended outcomes because they failed to engage important stakeholders or to remove important inconsistencies in the broader policy-framework (Bastos Lima et al., 2017; Rodriguez-Ward et al., 2018).

A major challenge to evaluation is that it remains relatively unclear what is meant by jurisdictional approach. “Heterogeneity of governance, cultural, and biophysical factors in jurisdictions as well as the changing context for jurisdictional programs has delayed conceptual development and creates a major challenge for retrospective evaluation. The sense of urgency to demonstrate the potential for landscape programs to contribute to global climate mechanisms often undermined careful documentation and clarification of how these programs work.” (Hovani et al., 2018).

This paper contributes to addressing this challenge by designing a theory of change to evaluate JAs that integrates existing empirical knowledge on JAs and middle-range theories from the literature on collective environmental governance.

The literature reviewed to write this paper is based on a selection of scientific articles extracted from a search in Scopus database¹, supplemented by a selection of key references from the gray literature².

Why Is a Theory of Change Needed to Evaluate Jurisdictional Approaches?

Evaluation Practices of Jurisdictional Approaches

Evaluation is commonly used to refer to impact evaluation. In this paper, we follow Patton (2018) and employ a broader definition of evaluation that also encompasses monitoring. Patton (2018) defines evaluation as a systematic process to determine merit, worth, value or significance of whatever is being evaluated (a program, policy, product, activity, etc.). Evaluation can serve different purposes (e.g., improve or reorient an intervention or inform decisions whether to continue, scale up or replicate an intervention) and can be performed through different approaches (Patton, 2008). Impact evaluation, which emphasizes causality between an intervention and its outcomes, is a particularly popular type of evaluation (Stern et al., 2012), including in the conservation sector (Baylis et al., 2016). However, strengthening results and impacts of an intervention can be achieved through other types of evaluation (Stern et al., 2012). For example, results monitoring can make significant contributions to supporting and reinforcing an intervention (Patton, 2008; Stern et al., 2012).

With this broad definition of evaluation in mind, we identified three types of evaluation that have been applied to JAs in the literature that we reviewed. Firstly, jurisdictional performance monitoring tools are currently gaining significant traction. The early call by Nepstad et al. (Nepstad et al., 2013a, b) to create such jurisdictional performance system was soon followed by the emergence of practical experiences including the Terpercaya initiative³ in Indonesia, the Landscale approach⁴, and the PCI (Produce, Conserve, Include) strategy in Mato Grosso, Brazil⁵. These tools generally list a number of indicators that are meant to be monitored in order to measure progress toward landscape or jurisdictional sustainability. These tools target multiple evaluation purposes. They help design more effective jurisdictional management practices; they provide relevant information to consumers, traders and investors willing to purchase or invest in sustainably produced commodities or to make green investment; and they can be used as criteria to design incentives, such as fiscal transfers, aimed at accelerating transitions toward sustainability. Overall, these strategies put a lot of emphasis on the identification of relevant, acceptable and easily available metrics to assess outcomes of JAs. However, they do not generate evidence on causal links between interventions and outcomes.

Secondly, we found a few attempts to evaluate the impact of JAs or conservation policies targeting the jurisdictional scale and aimed at reducing deforestation. Scholars usually rely on counterfactual approaches and methods such as matching (Cisneros et al., 2015; Sills et al., 2017, 2020) and synthetic controls (Sills et al., 2015; Roopsind et al., 2019) to evaluate the impact of jurisdictional scale interventions (REDD+ incentives, blacklisting of municipalities in Brazil). Impact evaluation is mostly used for summative purposes, i.e., to check whether money has been spent for something and to subsequently inform decisions to replicate or scale up such interventions (Ferraro, 2009; Sills et al., 2017). Counterfactual approaches provide credible evidence for causality between an intervention and an outcome because they are based on the construction of a realistic comparison group that represents what would have happened if the target group of the intervention had not received the intervention. However, they generally fall short in providing evidence for the underlying mechanism and impact pathways, as they tend to focus on longer-term outcomes (e.g., deforestation, well-being, revenues, etc.). Furthermore, finding appropriate counterfactuals for evaluation objects such as JAs that are implemented at large scales remains difficult, particularly because of the small pool of potential controls (Sayer et al., 2017; Roopsind et al., 2019).

Finally, we identified a number of papers that evaluate the underlying processes of jurisdictional REDD+ with the aim to identify challenges, pitfalls and in turn improve outcomes (Ravikumar et al., 2015; Sanders et al., 2017; Rodriguez-Ward et al., 2018). These studies provide evidence for the influence of various governance aspects on jurisdictional REDD+ outcomes. Evidence for causal mechanisms is based on the triangulation of various data sources, including in particular semi-structured interviews with a large number of stakeholders and relevant secondary data. However, empirical knowledge about how JA work in practice and under what conditions remain scarce.

Prerequisites for Implementing Theory-Driven Evaluations of JA

There is a wide consensus in the evaluation literature regarding the importance of using theories of change to evaluate interventions (Astbury and Leeuw, 2010; Leeuw and Donaldson, 2015), particularly for complex interventions (Rey et al., 2011). A theory of change in evaluation generally articulates causal linkages between interventions and their desired effects. It also shows “how and why programs work (or fail to work) in different contexts and for different program stakeholders” (Astbury and Leeuw, 2010). The use of theories of change improves all three types of evaluation found in the reviewed literature on JA. For example, Miteva et al. (2012) argues that one of the major drawbacks of current impact evaluation studies is that “the empirical work is disconnected from theories that describe how the interventions affect outcomes.” This limit affects both the internal validity i.e., the definition of the counterfactual (high risk of omitted variables) and external validity i.e., to what extent the results of the evaluation can be generalized (Miteva et al., 2012; Baylis et al., 2016). Theories of Change are also useful for the design of monitoring tools aimed at measuring the progress of conservation actions toward an intended goal (Margoluis et al., 2013).

We argue that, in the case of JAs, there are at least two components of theories of change that need to be strengthened. Firstly, there is a need to clarify the causal sequences of effects associated with the implementation of JA. We found limited evidence in the reviewed literature that JA empirical cases developed a clear vision on how things are supposed to unfold as a result of JA implementation in order to achieve deforestation reduction at the jurisdiction scale. In particular, there is no evidence that the most common jurisdictional performance monitoring tools are based on a clear theory of change. For example, Terpercaya indicators are “based on Indonesian laws and regulations and are aligned with the Sustainable Development Goals and the Paris Agreement on climate change.” Links between the different indicators and the underlying assumptions are not explicit. Similarly, the PCI dashboard compiles several long-term indicators that are not explicitly embedded in a theory of change linking it with interventions. We found only one study that attempts to define theories of change for JA based on extended empirical knowledge gained from studying and implementing such approaches (Hovani et al., 2018). In this study, TNC produced a Theory of Change that clearly articulates intermediary outcomes showing “how improved cross-sector collaboration can translate into improved landscape performance and define enabling and hindering conditions for a number of causal links” (Hovani et al., 2018).

Secondly, there is a need to identify the specific mix of interventions implemented as part of JAs. In the early days of the literature on JAs, Nepstad et al. (2013b) advocated for strengthening linkages between three main types of interventions aimed at inducing emissions reduction from deforestation at the jurisdictional scale, namely REDD+, domestic policies and finance, and value chain initiatives. Recently, the gray literature has converged on the term “Jurisdictional Programs (JP),” defined as a range of multi-stakeholder interventions applied at sub-national levels in order to achieve lasting, jurisdiction-wide improvements to natural resource management by catalyzing collaborative action by a group of stakeholders working with local government to institutionalize improved land governance and land use practices (Paoli et al., 2016; Hovani et al., 2018). Interestingly, these JPs encompass interventions that are generally not included or are marginally represented in classical typologies of conservation intervention (Lambin et al., 2014; Puri and Nath, 2016; Pirard et al., 2019). JPs typically focus on strengthening local governance by strengthening the network of actors, improving accountability, investing in knowledge development, etc. The other specificity of JPs is that it intends to cover a large scope of actors and policy domains. Furthermore, they span over long-time periods and are characterized by the emergence of new goals and interventions during the implementation phase. As a result, identifying the mix of interventions associated with a given JA is particularly challenging. Therefore, a theory of change, even if built retrospectively, can help describe such a system of interventions and justify the coherence between its components (Craig et al., 2008).

In order to extend ongoing work on JA theories of change and definition of JPs, we build on Leeuw and Donaldson’s (2015) two types of theories in evaluation. According to these authors, the first type of theory “helps to understand the theories that underlie policy makers’ and stakeholders’ programs and policies” while the second type focuses on external, scientific theories that are capable of explaining the consequences of policies, programs, and evaluative actions (Leeuw and Donaldson, 2015). This second type of theory-building adds crucial insights about the mechanisms through which, and contexts under which, the causal sequence operates and allows putting into perspective the subjective insights provided by the professionals involved in these interventions (Leeuw and Donaldson, 2015). Building on the idea that both types need to be combined to draft meaningful theories, the rest of the paper presents an attempt to integrate current empirical knowledge on JA and middle-range theories from the literature on collective environmental governance into a generic theory of change.

A Proposal for a Generic Theory of Change for JA

In this section, we describe a generic theory of change chain for JA (Figure 1). We describe this theory of change from right to left, i.e., from the long-term impact to the short-term intermediate outcomes.

Figure 1. Generic theory of change chain for JA drafted with Miradi software.

We consider the narrower cases of JAs, which aim at reducing deforestation as their long-run goal. However, following Hovani et al. (2018), we consider that this long-term impact is not a direct result of JAs. We argue that the direct outcome of JA is the formalization of a consistent and locally adapted framework of operational and collective rules (section “Formalization of a Framework of Rules”). We then suggest that formulating such a framework of rules necessarily involves multi-stakeholder collaboration across sectors and levels, and social learning (section “Effective Collaborative Processes”). In turn, both social learning and multi-stakeholder collaboration will only occur under some conditions (section “Conditions for Social Learning and the Multi-Stakeholder Collaboration”). We finally point to a number of intervention strategies that could comprise a jurisdictional program (section “Interventions”).

Formalization of a Framework of Rules

Operational Rules

The main contribution of JAs to reducing pressure on forests is the formalization of a consistent framework of operational rules such as incentives, policies, laws, and practices (Denier et al., 2015). These rules directly influence how decisions on natural resource management are taken (Ostrom, 2011). For example, in Acre, one of the flagship cases of Jurisdictional REDD+ that was integrated into state law, conditional incentives for ecosystem service conservation for indigenous peoples, local communities, smallholders and other land users (Da Conceição et al., 2015) were integrated into a policy mix that built on decades of forest-based development (Schmink et al., 2014).

The emergence of local operational rules as well as the main drivers of deforestation are influenced by the characteristics of the broader economic and political context outside the jurisdiction itself. It requires that inconsistencies in the broader national and international legal and policy framework are removed. In Acre and the Yucatan Peninsula, two jurisdictional REDD+ cases show that the success of sub-national initiatives “is hindered by the persistence of business-as-usual incentives stemming from the national level, and over which sub-national programs have little control” (Bastos Lima et al., 2017). It might also require linking the jurisdiction with existing higher-level incentives and rules for low emission development (Boyd et al., 2018).

Collective Rules for Land-Use

Achieving positive outcomes is contingent on stakeholders agreeing on a number of collective rules, i.e., rules that determine how operational rules are designed or revised (Ostrom, 2011). It includes a shared vision of goals for deforestation reduction, clarified rights and responsibilities, particularly tenure rights, a land-use plan clarifying underlying trade-offs, monitoring and reporting systems and grievance and conflict resolution mechanisms. Prior to creating its incentive program, Acre defined in 2007 an Ecological and Economic Zoning Plan that divides the province into geographic zones, regulates economic activities in previously deforested areas, and provides the basis for sustainable management in areas that remain forested (Secretaria and Ambiento, 2010). Such collective rules are amongst the progress indicators of major landscape governance assessment tools. For example, the Terpercaya indicators include sustainable land-use planning, complaint mechanism, customary rights recognition. The “Sustainable Landscapes Rating Tool” defines land use plan/zoning, clarity of land and resource tenure and use rights, monitoring and reporting systems, sustainable landscape goals as key enabling conditions for jurisdictional policies and governance that enable sustainable landscapes⁶.

More, to be actually enforced, locally defined land-use plans or tenure and use rights often require central government’s recognition to become effective. For example, the Federal Government of Brazil approved the Acre’s zoning plan before it was used to support the creation of the incentive program in the state (Stickler et al., 2018).

Effective Collaborative Processes

In the JA context, an effective process of rule creation should involve multi-stakeholder collaboration across sectors and levels, and social learning to formulate rules that are compatible with everyone’s interests and that are context-specific (Paoli et al., 2016; Fishman et al., 2017; Hovani et al., 2018).

Multi-Stakeholder Collaboration

Multi-stakeholder collaboration across sectors and levels is well-emphasized in the subset of the JA literature reviewed (Paoli et al., 2016; Fishman et al., 2017; Rodriguez-Ward et al., 2018). It can be formalized through the creation of a multi stakeholder platform, but should anyway encompass the representatives of the various interest groups of a jurisdiction (Hovani et al., 2018). Fundamentally, cross-scale and cross-border collaboration is critical to engender collective rulemaking in this type of setting. Direct causes of environmental degradation–deforestation in particular–are the result of actions taken by various land users while solutions to solve this problem are often under the authority of different actors with a high degree of autonomy (Cash et al., 2006; Folke et al., 2007). Fishman et al. (2017) argue that the process of land-use planning in the context of JAs “needs to be participatory to ensure that the right delineations are selected, drawing on local expertise, and to ensure buy-in from stakeholders who will be expected to abide by the plan.” In a JP setting, engagement of subnational government leaders is particularly crucial to be able to institutionalize change. Indeed, they are in a position to obtain the endorsement of changes in laws, plans and regulatory procedures, to strengthen law enforcement and to influence other key stakeholder’s propensity to engage in JAs. Besides, the exclusion of important players at grass-root level is presented as a significant risk for JA success (Fishman et al., 2017; Rodriguez-Ward et al., 2018). In Madre de Dios, another case of jurisdictional REDD+ in Peru, effective integration of Non-Governmental Organizations (NGOs) and indigenous groups in decision-making was hindered by power imbalances and politics and proved to be an obstacle to the emergence of integrated landscape governance (Rodriguez-Ward et al., 2018).

Social Learning

Social learning is a core factor conducive to the emergence of adapted interventions and rules in JAs. Social learning is defined as a process characterized by a change in understanding in the individuals involved and within wider social units or communities of practice that occur through social interactions and processes between actors within a social network (Reed et al., 2010). Social learning is particularly important in the case of JAs, because management decisions and plans have to be made in a context of high complexity and uncertainty and because JAs entail heterogeneity of actors and institutions in terms of power, needs, aspirations and interests (Armitage et al., 2009). TNC stresses that it is through social interactions that stakeholders can develop shared understanding of what is happening in the landscape, clarify their own goals and interests, learn about those of others, and strengthen trust. Eventually, this helps develop greater clarity on high-leverage interventions that are compatible with everyone’s interests (Hovani et al., 2018).

Conditions for Social Learning and the Multi-Stakeholder Collaboration

Emergence of Collaboration

We identified in the JA literature reviewed a number of enabling/hindering conditions influencing the emergence of collaboration. In order to clarify the underlying assumptions, we build on literature on the Institutional Rational Choice approach. Within this type of approach, rational actors will engage in collaborative solutions to common-pool resources or public good dilemmas if the benefits of the outcomes for them outweigh the transaction costs of collaboration (Schlager and Blomquist, 1996; Ostrom, 2007). In turn, the characteristics of the social-ecological system (i.e., the context) influence the level of costs and benefits of engaging into collaboration and subsequently the emergence of collaborative arrangements (Schlager and Blomquist, 1996; Ostrom, 2007). In Table 1, we specify the enabling conditions that are likely to influence JAs, the corresponding hypotheses about underlying mechanisms, and key references in the subset of Institutional Rational Choice and JA literature reviewed.

Table 1. Key context-mechanism hypotheses for the emergence of collaboration in the case of Jas.

Conditions for Social Learning

The literature on adaptive governance of social-ecological systems also identifies a number of conditions of social learning and assumptions about the underlying mechanisms. We use this material to summarize the main conditions of social learning identified in the reviewed literature on JAs related to reducing deforestation, and clarify the hypothesis about the underlying mechanisms (Table 2).

Table 2. Key context-mechanism hypotheses for social learning in the case of Jas.

Interventions

Clarifying these second-tier intermediary outcomes provides guidance to refine or at least justify the objectives and strategies of jurisdictional programs. In Table 3, we present a number of strategies found in the JA literature reviewed (Paoli et al., 2016; Fishman et al., 2017; Hovani et al., 2018; Rodriguez-Ward et al., 2018) that different stakeholders, including stakeholders that are external to the jurisdiction, can undertake to enhance collaboration and social learning and, in turn, reduce deforestation at the jurisdictional level. These strategies are aimed at creating the appropriate conditions for engaging jurisdictional stakeholders in collaboration around the management of land-use and for sustained learning.

Table 3. Key intervention strategies for a jurisdictional program.

Conclusion/Discussion

The current enthusiasm for jurisdictional approaches to reduce deforestation is mainly based on the promise of effectiveness gains, which are yet to be verified. In this paper, we contend that further clarification of the JA concept would help its evaluation. We suggest that current evaluation practices of JAs would be strengthened if they were based on a theory of change clarifying the causal linkages between the interventions associated with a given JA and its effects.

Our contribution is a generic theory of change based on the integration of select empirical knowledge on JAs with middle-range theories from the literature on collective environmental governance. Our generic theory of change for JAs is articulated around two central intermediary outcomes–namely the emergence of collaboration and social learning–in order to achieve deforestation reduction through the development of a coherent set of locally devised collective and operational rules. The literature on social learning and on the emergence of collaborative arrangements for the management of natural resources proved to be helpful to clarify assumptions underlying the links between second and third-tier outcomes. We also highlight that the effectiveness of JAs depends on a number of local but also external conditions. A challenge for JAs is also to remove the external contradicting signals (e.g., commodity markets, national, and international policies) that are still fundamental in driving deforestation (Curtis et al., 2018). As such, a JP would need to consider processes not only at the jurisdictional level, but also beyond.

Our generic theory of change has several limitations that are intimately linked to the fact that it is rather general and based on quite a limited pool of documentary resources and empirical experiences. Firstly, the different assumptions attached to the conditions for social learning and collaboration may not be valid in every context and may need to be specified. For example, “social learning may take place in the absence of any planned participatory process” (Reed et al., 2010). Similarly, some studies have pointed to the limits of mainstream monitoring and evaluation approaches and scientific knowledge to serve pro-poor and pro-environmental change processes (Guijt, 2008; Koontz et al., 2015). Specific types of monitoring knowledge may have to be generated or scientific knowledge may need to be turned into usable knowledge in order to address specific local knowledge needs. Secondly, some conditions for collective action or social learning that have not been presented in this paper may also prove to be of crucial importance in some contexts. For example, the literature on common pool resources shows how important the resource characteristics are in explaining the emergence of successful collective management (Ostrom, 1990; Agrawal, 2001).

These limits call for further empirical research on the theory of change of JA. This could be achieved by improving our understanding of stakeholders’ perspective of JA theory (type I theory in Leeuw and Donaldson, 2015) using participatory methods such as scenario-building or visioning exercises and participatory Theory of Change exercises (Romero et al., 2017; Sayer et al., 2017; Qiu et al., 2018). This work can also be completed by scientific studies aimed at testing hypotheses derived from the theoretical literature (type II theory in Leeuw and Donaldson, 2015). This kind of process can be relatively costly, and thus requires sufficient awareness about the importance of evaluation, as well as a minimum level of willingness to pay amongst implementing and donor agencies. It is a necessary investment, though, if we want to more accurately evaluate the conditions of effectiveness and realize the actual potential of JAs in contributing to reducing deforestation and promoting jurisdictional sustainability.

Author Contributions

CC and MGP discussed the original idea of the manuscript. CC wrote the manuscript and lead the revisions. JR and MGP provided insightful comments and edited the language and scientific content of the manuscript at all stages. All authors contributed to the article and approved the submitted version.

Funding

This research was part of Priorities 5 and 18 of the CGIAR (Consultative Group on International Agricultural Research) Research Program on Forests, Trees and Agroforestry (FTA, http://foreststreesagroforestry.org). The funding partners that have supported this research include the CGIAR Trust Fund (www.cgiar.org/funders), CIRAD (Centre International de Recherche Agronomique pour le Développement) through its CRESI (Creativity and Scientific Innovation) funding program, and CIFOR (Center for International Forestry Research) through its Global Comparative Study on REDD+ (www.cifor.org/gcs) funded by the Norwegian Agency for Development Cooperation (Norad).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer MM declared a shared affiliation with one of the authors, JR, to the handling editor at time of review.

Acknowledgments

We finally thank the three reviewers for their insightful comments.

Footnotes

1. ^ We used the following request: TITLE-ABS-KEY [(jurisdiction^∗ AND redd+) OR “jurisdictional approach^∗”) AND (EXCLUDE (SUBJAREA, “MEDI”)]. We selected articles that dealt with the evaluation of JAs.
2. ^ We selected the references that drew lessons based on empirical experience or analysis and that aim to improve JA practice.
3. ^ http://www.euredd.efi.int/publications/demonstrating-and-promoting-district-level-sustainable-commodity-production
4. ^ https://www.landscale.org/
5. ^ http://pcimonitor.org/
6. ^ https://www.climate-standards.org/sustainable-landscapes-rating-tool/

References