Structuring and evaluating decision support processes to enhance the robustness of complex human–natural systems
Introduction
The debate on global environmental and societal change has shifted from problems to solutions in recent years (Schellnhuber et al., 2011). A globally prominent example is the UN 2030 Agenda for Sustainable Development with its urgent call to action for achieving 17 Sustainable Development Goals (SDGs) (UN, 2015), aiming at multiple environmental and socio-economic aspects, such as decarbonising energy systems, conserving biodiversity, reducing poverty, and improving gender equality. To achieve such diverse goals, decision-makers need to make robust decisions insensitive to well-characterised as well as deep uncertainties (Ben-Haim, 2006; Lempert et al., 2003) that are characteristics of complex and coupled human–natural systems. Deep uncertainty emerges from the limited and contested knowledge among stakeholders about interacting human–natural systems and their boundaries, the state (e.g., the likelihood) of key drivers of these systems (e.g., population growth, water availability, energy demand), and the relative importance of the outcomes (e.g., trade-offs between goals for economic development and climate actions) (Khatami et al., 2019; Kwakkel et al., 2016a; Singh et al., 2015). The shift towards robust decisions requires careful accounting of candidate costs or regrets associated with highly precautionary or risk-averse decision alternatives (Dessai and Hulme, 2007; Lincke and Hinkel, 2018; Small and Xian, 2018; Weaver et al., 2013). We aim to develop a unified framework that can inform robust inferences in complex human–natural systems by classifying, sequencing, and evaluating alternative framings of decision support. We specifically focus on model-based decision support under different degrees of uncertainty (from well-characterised to deep uncertainty) where a model (i.e., conceptual or quantitative) of the system of interest is used for the evaluation of assumptions and hypotheses (Walker, 2000).
Decision support in this context requires a sequence of interactive methodological choices about how to set the problem context (e.g., deep or well-defined uncertainty?), how to frame the problem (e.g., limited scenario narratives or many quantitative scenarios?), how to evaluate candidate decisions (e.g., model simulation or qualitative evaluation?), and how to drive robust inferences and provide decision recommendations (Tsoukiàs, 2008). Alternative combinations of the variety of choices made by analysts can lead to fundamentally different outcomes for the systems of focus (see Kwakkel, 2017). Decision support is also a subjective and social process, involving various stakeholders who directly or indirectly influence the methodological choices underlying any analysis (Voinov et al., 2014). Within this subjective process, methodological choices can be influenced deliberately (e.g., through self-interest) or unintentionally (e.g., through cognitive bias or ignorance) (Argyris and Schön, 1978; Lahtinen and Hämäläinen, 2016; Mayer et al., 2017). For example, in new projects, analysts often tend to favour methods and approaches with which they are familiar and have used previously, potentially in unrelated contexts (Lahtinen et al., 2017). These traits can influence the evaluation of robust decisions by making one particular methodological choice (e.g., deterministic simulation) to be selected over other—possibly more effective—ones (e.g., a robustness analysis which considers deep uncertainty). The risks of biased methodological choices are in sub-optimal, or even misleading, decision implications for the project (Kasprzyk et al., 2009; Lempert and Collins, 2007; Oddo et al., 2017; Singh et al., 2015).
Several studies have reviewed and classified common methodological choices in robustness-focused decision support frameworks (Dittrich et al., 2016; Herman et al., 2015; Kwakkel and Haasnoot, 2019; Maier et al., 2016). However, they are limited in addressing how researchers and practitioners should (a) make the choices that are most appropriate given different levels of knowledge, (b) become critically aware of the human factors that influence their methodological choices, (c) evaluate the combined consequences of these choices and factors on planning outcomes, and (d) take measures to tackle their negative consequences. We clarify sequences and interdependencies of methodological choices under the influence of critical human factors (e.g., biases, beliefs, heuristics, values) from psychology and cognitive sciences. We articulate how alternative candidate options for undertaking decision support processes (termed decision support paths) emerge from the cumulative effects of choices made at different stages of the process (termed decision forks). This enables us to analyse how alternative decision support paths may lead to a diversity of final inferences that are sensitive to the path taken (i.e., path dependency). We also go beyond existing taxonomies in the literature, which mostly focus on choices of analytical components (e.g., how to generate decisions and scenarios), and review new methodological steps for framing and evaluating the decision support processes themselves.
The articulation of methodological choices at decision forks, the resulting decision support paths, their path dependency, and their influential human factors contributes to policy discussions around assessing global environmental and societal sustainability and change in three ways:
- •
First, it can pave the way for the exploration and the deliberative improvement of robust decision-making by illuminating the presence of alternative—sometimes less well-understood—rival paths to be considered (Quinn et al., 2017; Saltelli and Giampietro, 2017; Wirtz and Nowak, 2017). Recognising and identifying rival paths leads to awareness about the variety of potential alternative ways of informing decision-making depending on the context without being locked into one dominant—but not necessarily relevant—implementation of decision support.
- •
Second, the transparent articulation of available methodological choices and potential paths helps to demystify the decision support process and to facilitate a better understanding of the overall effects of methodological choices on final inferences. Carefully documenting decision support paths is critical for ensuring that decision recommendations are supported by credible evidence, and therefore are defensible (Cash et al., 2003; Cooke, 1991).
- •
Third, discussion of the impact of influential human factors helps researchers and practitioners to be critically aware of the effects of mental and behavioural aspects on the decision support process and their range of consequences for conclusions. It also signifies the importance of recognition, evaluation, and management of influential human factors in research projects to improve the effectiveness of outcomes.
Section 2 presents the four conceptual bases of the current work. Section 3 explains methodological choices and human factors and their potential influence throughout the decision support process. Section 4 discusses frontier challenges, opportunities, and recommendations for future decision support innovations.
Section snippets
Conceptual basis
This section briefly introduces the four conceptual pillars for this article. We use concepts drawn from: a) constructive decision aiding to set out a generic description of the decision support process and different forks at which choices need to be made; b) a taxonomy of robustness frameworks to articulate methodological choices at decision forks; c) the path perspective to demonstrate the way in which particular methodological choices can lead to specific implementations of decision support;
Rival decision support paths
Rival decision support paths can emerge from the succession of methodological choices at different decision forks, their path dependency, and the effects of influential human factors (e.g., biases, values). Fig. 3 shows an overview of these decision forks with their potential methodological choices. The interdependencies between decision forks create cumulative impacts on the methodological design and on the final planning outcomes (path dependency). For example, limited stakeholder knowledge
Challenges, opportunities, and future research
Supporting scientific robust decision-making in complex human–natural systems undergoing global change becomes exceedingly difficult because of tensions between competing and sometimes conflicting frames of problems (e.g., competing objectives, high-dimensional and stochastic nature of systems) that are formulated under different stakeholder assumptions (Bosomworth et al., 2017). Stakeholder participation and engagement have been recognised as a determinant for achieving translational real
Declaration of competing interest
The authors whose names are listed for the submitted manuscript entitled “Structuring and evaluating decision support processes to enhance the robustness of complex human–natural systems” have NO conflict of interests in the current work.
References (185)
- et al.
Seeking sustainability discourses with Q methodology
Ecol. Econ.
(1999) - et al.
Robust optimization–a comprehensive survey
Comput. Methods Appl. Mech. Eng.
(2007) - et al.
What's the problem in adaptation pathways planning? The potential of a diagnostic problem-structuring approach
Environ. Sci. Policy
(2017) - et al.
Thinking inside the box: a participatory, computer-assisted approach to scenario discovery
Technol. Forecast. Soc. Chang.
(2010) - et al.
Assessing the robustness of adaptation decisions to climate change uncertainties: a case study on water resources management in the East of England
Glob. Environ. Chang.
(2007) - et al.
A survey of decision-making approaches for climate change adaptation: are robust methods the way forward?
Ecol. Econ.
(2016) - et al.
Including robustness considerations in the search phase of many-objective robust decision making
Environ. Model. Softw
(2018) - et al.
A model-based analysis of biomethane production in The Netherlands and the effectiveness of the subsidization policy under uncertainty
Energy Policy
(2015) - et al.
Development of a discrete event simulation model for evaluating strategies of red blood cell provision following mass casualty events
Eur. J. Oper. Res.
(2018) - et al.
A new analytic method for finding policy-relevant scenarios
Glob. Environ. Chang.
(2007)
The diversity of socio-economic pathways and CO2 emissions scenarios: insights from the investigation of a scenarios database
Environ. Model. Softw
Dynamic adaptive policy pathways: a method for crafting robust decisions for a deeply uncertain world
Glob. Environ. Chang.
Fit for purpose? Building and evaluating a fast, integrated model for exploring water policy pathways
Environ. Model. Softw
Designing a monitoring system to detect signals to adapt to uncertain climate change
Glob. Environ. Chang.
An open source framework for many-objective robust decision making
Environ. Model. Softw
A methodological framework to support the initiation, design and institutionalization of participatory modeling processes in water resources management
J. Hydrol.
Behavioural issues in environmental modelling – the missing perspective
Environ. Model. Softw
The threat of weighting biases in environmental decision analysis
Ecol. Econ.
On the importance of behavioral operational research: the case of understanding and communicating about dynamic systems
Eur. J. Oper. Res.
An exploratory approach for adaptive policymaking by using multi-objective robust optimization
Simul. Model. Pract. Theory
Agent-based modelling and socio-technical energy transitions: a systematic literature review
Energy Research & Social Science
Designing monitoring arrangements for collaborative learning about adaptation pathways
Environ. Sci. Policy
Scenario generation using adaptive sampling: the case of resource scarcity
Environ. Model. Softw
Many-objective de Novo water supply portfolio planning under deep uncertainty
Environ. Model. Softw
Many objective robust decision making for complex environmental systems undergoing change
Environ. Model. Softw
Climatic or regionally induced by humans? Tracing hydro-climatic and land-use changes to better understand the Lake Urmia tragedy
J. Hydrol.
Comparing state-of-the-art evolutionary multi-objective algorithms for long-term groundwater monitoring design
Adv. Water Resour.
The Exploratory Modeling Workbench: an open source toolkit for exploratory modeling, scenario discovery, and (multi-objective) robust decision making
Environ. Model. Softw
Comparing robust decision-making and dynamic adaptive policy pathways for model-based decision support under deep uncertainty
Environ. Model. Softw
Path dependence and biases in the even swaps decision analysis method
Eur. J. Oper. Res.
Why pay attention to paths in the practice of environmental modelling?
Environ. Model. Softw
Model validation in operations research
Eur. J. Oper. Res.
Identifying and evaluating robust adaptive policy responses to climate change for water management agencies in the American west
Technol. Forecast. Soc.
Economically robust protection against 21st century sea-level rise
Glob. Environ. Chang.
Linking science with environmental decision making: experiences from an integrated modeling approach to supporting sustainable water resources management
Environ. Model. Softw
Simulation optimization: a review of algorithms and applications
Ann. Oper. Res.
Organizational Learning: A Theory of Action Perspective
Climate change communication: what can we learn from communication theory?
Wiley Interdisciplinary Reviews: Clim. Change
Exploratory modeling for policy analysis
Oper. Res.
Tools and techniques for developing policies for complex and uncertain systems
Proc. Natl. Acad. Sci.
Computer-assisted reasoning
Comput. Sci. Eng.
Is Multiobjective Optimization Ready for Water Resources Practitioners? Utility's Drought Policy Investigation
Info-Gap Decision Theory: Decisions under Severe Uncertainty
Building a values‐informed mental model for new orleans climate risk management
Risk Anal.
An actor‐oriented approach for strengthening research and development capabilities in natural resource systems
Public Adm. Dev.
Risk-based water resources planning: Incorporating probabilistic nonstationary climate uncertainties
Water Resour. Res.
MGA: a decision support system for complex, incompletely defined problems
IEEE Transactions on Systems, Man, and Cybernetics
Decision Scaling: Linking bottom‐up vulnerability analysis with climate projections in the water sector
Water Resour. Res.
What to do when Stakeholders matter
Public Manag. Rev.
Knowledge systems for sustainable development
Proc. Natl. Acad. Sci.
Cited by (56)
Designing a pattern language to enhance model composability and reusability: An example with component-based probabilistic models
2023, Environmental Modelling and SoftwareKnowledge co-production for decision-making in human-natural systems under uncertainty
2023, Global Environmental ChangeUnderstanding robustness in multiscale nutrient abatement: Probabilistic simulation-optimization using Bayesian network emulators
2022, Journal of Cleaner ProductionFirst, do no harm - Missing data treatment to support lake ecological condition assessment
2022, Environmental Modelling and SoftwareViability, efficiency, resilience and equity: Using very diverse indicators to deal with uncertainties of future events
2022, Environmental Science and Policy