Shedding light on the usability of ecosystem services–based decision support systems: An eye-tracking study linked to the cognitive probing approach

Highlights

• Results of the study provide design recommendations for representing ES information.

• Types of representation of the information influence perception and cognitive processes.

• Participants interpreted and used the information for ES-based decision-making differently.

• Detail and scale between applied ES information and reasoning is correlating.

Abstract

The requirements for communicating ecosystem services (ES) information often are not considered in operationalizing ES concepts. In particular, the heterogeneous uses of ES require different functionalities and qualities for the information provided, which must be considered when processing ES data into different types of information. The relevant factors that influence the usability of ES information include the users’ knowledge and cognitive ability as well as case study–specific factors. This missing knowledge can affect the transformation of the ES concept into practice, thus preventing the use of ES for further development or for transformation to sustainable management. Providing information that is relevant and useful for decision-making thus depends on understanding potential users’ demands and their cognitive processes involving the information in making decisions.

In this contribution, we present the evaluation of specific design features of a prototype ES decision support system assessed in an eye-tracking experiment. The study was conducted with more than 100 participants who were split into two groups. The participants in both groups had a background in spatial planning but differed in their connection to the case study region. The tool presented various GIS-based modeled land-use scenarios driven by a new spatial planning policy recently adopted in Switzerland that lead to various impacts on ES in the region. The ES information was shown with additional land-use indicators as well as information about the landscape aesthetic in landscape visualizations. The results show that there were significant differences between the participants in the way they perceived, interpreted and used the information for ES-based decision-making tasks. We also identified critical key factors defining the types of representation of the information that influence perception and cognitive processes.

In summary, the results of the study provide design recommendations for representing ES information based on the intended use and identify critical representation features that could potentially influence the perception of ES information.

Introduction

In 2011, the European Commission (EU) adopted the EU biodiversity strategy to halt the loss of biodiversity and Ecosystem Services (ES) in the EU by 2020 (European Commission, 2012). The EU strategy targets public awareness of ES issues in addition to establishing education and communication campaigns as well as developing instruments for more effective ES management and providing information on ES. These targets are crucial elements of sound decision-making and therefore call for an improvement in and implementation of ES information in spatial planning tools and processes to provide ES-based reasoning and communication to stakeholders and the public. At the same time, the existing working group on Mapping of Ecosystems and their Services in the EU and its Member States (MESEU) has been investigating the best practices for supporting the improved implementation of ES information in policy and decision-making (MESEU, 2014). To achieve the strategy targets by 2020, information on the ES provided at the local scale is indispensable for implementing the information in spatial planning. In terms of communication strategy, ES information can be provided in a wide range of different types and scales, but clear guidelines on which types or scales are suitable for conveying this information to various types of users is lacking (Klein et al., 2015).

A new trend is to provide the public with spatially explicit environmental information–for instance, information on provision of ES–via streamlined, easy-to-use and often web-based GIS platforms (e.g., www.ecometrica.com). Some of these platforms are specifically designed to provide relevant information in decision-making processes or to allow exploration of future scenarios (e.g., Wissen Hayek et al., 2015, Grêt-Regamey et al., 2013). Such platforms are also known as planning support systems or decision support systems (DSSs). In landscape and urban planning such DSS can contribute to support sound decisions that account for sustainable use of ecosystems and their providing services. The trend of such DSS emergence has been stimulated by modern information and communication technologies and policy strategies, such as worldwide access to broadband Internet (also an EU initiative; European Commission, 2015a). Furthermore, EU policy aspires to provide cross-national spatial information: For example, the EU’s Infrastructure for Spatial Information in the European Community (INSPIRE) aims at establishing common data typologies for transnational environmental assessments and environmental policies (European Commission, 2015b). In addition, national laws for the provision of and public access to spatial information were passed in recent years, for example, in Germany (BMUB, 2012) and Switzerland (GeoIG, 2007). With these regulations, access to environmental and ES information can also be enabled, allowing potentially easier use and implementation of administrative information in a DSS, which would facilitate transparency, credibility and legitimacy, as previous studies have shown (e.g., Wissen Hayek et al., 2015; Ruckelshaus et al., 2015; Pettit et al., 2011, Cash et al., 2003).

Empirical studies in spatial decision-making have shown that the amount of information affects the quality of the decisions (e.g., Jankowsky and Nyerges, 2001; Jelokhani-Niaraki and Malczewski, 2014). For example, as the number of alternative locations or criteria available in the decision-making process increases, stakeholders also need an increasingly deeper understanding of the relations and dependencies of the locations or criteria to assess and prioritize them (Jelokhani-Niaraki and Malczewski, 2015). Furthermore, recognition of relations and dependencies becomes more difficult, and users then tend to simplify their decision-making processes to avoid high cognitive demands for examining the information. Consequently, low-quality decision-making and a low level of consensus between decision-makers frequently occur (Jelokhani-Niaraki and Malczewski, 2015). Although the relevance of information integrated in a DSS facilitates the transparency, credibility and legitimacy of decision-making (Ruckelshaus et al., 2015), the best methods for representing information so that the users’ decision-making process is most effectively supported and the level of information required for high-quality decisions remain unclear.

In general, to communicate is to transmit information so that it is understood and, typically, used to guide action. For environmental information, the relation and interaction between different environmental criteria make successful communication a complex, multifaceted task. This complexity is further increased by spatial information, which makes comprehensive understanding and, therefore, effective communication more difficult (Mors et al., 2010). The initially communicated environmental information hinders easy information transfer because of the multifaceted effects on other environmental criteria. Especially, the communication of combined environmental and spatial information can lead to complex socio-psychological interactions (Mors et al., 2010), including emotional reactions if recipients are personally affected or have a relation to an affected place (e.g., Veríssimo and Campbell, 2015, Rogge et al., 2011). As previous studies have shown, recipients can often cognitively link the communicated environmental criteria to landscape aesthetics (e.g., Junker and Buchecker, 2008). Such an extended perspective of non-DSS-included information (as they would be supported by landscape visualizations) can be based on either experience or knowledge of the place. These reactions can be identified over the course of participative landscape planning approaches in which stakeholders react and interact with provided information (e.g., Celio et al., 2014, Höppner et al., 2007). In contrast, a lack of information or criteria that are used for reasoning can affect the trust or confidence in a DSS, as there is a lack of completeness. Disinterest in participation or dissatisfying communication might be the consequence (Höppner et al., 2007). Most notably, not only the detail, comprehension and amount of information (e.g., indicators, criteria and localities) influence user emotions and behavior, but also the design of the presented information affects cognition and therefore the reasoning processes (Russo et al., 2014). Consequently, understanding the information requirements of DSS users can result in more comprehensive and improved communication and thus more effective and efficient decision-making due to the transparency, credibility and legitimacy of the information integrated in a DSS (Wissen Hayek et al., 2015, Ruckelshaus et al., 2015, Pettit et al., 2011). In summary, to determine how to provide the most effective and efficient information for users, two main aspects must be addressed: how to communicate environmental and ES information comprehensively and how to represent such information. Knowledge of these aspects can avoid negative effects such as a loss of trust and confidence, or emotional reactions that prevent an objective examination of the information (Pettit et al., 2011). Especially for DSSs, appropriate communication and presentation of information are important to support users with relevant and needed information in their personal decision-making strategy (Jelokhani-Niaraki and Malczewski, 2015; Vessey, 1991, Vessey and Galletta, 1991).

Novel techniques such as eye tracking (ET) make it possible to record humans’ gaze and, thus, to research visual behaviors in a natural setting. With this technique, we can investigate how DSS users use information and apply a DSS. ET has been proven to be a helpful technique in user research, especially for the evaluation of visual stimuli in practical applications. With ET, the length and frequency that users look or gaze at particular areas of interest (AOIs) can be determined (Duchowski, 2007, Holmqvist et al., 2011). The position of the gaze is typically expressed using screen coordinates (i.e., pixels). From these basic screen coordinate measurements, various gaze metrics are derived in relation to the stimuli (screen display), such as the fixation duration or dwell time (i.e., how long a gaze is fixed on a certain AOI), fixation count (i.e., how often the gaze revisits an AOI), number of revisits of the AOIs and scan-path characteristics (e.g., length and speed of eye movements; Ooms et al., 2014). Although a new technique, ET has already been applied in many research fields, such as software engineering (e.g., usability tests; e.g., Jacob and Karn, 2003, Nivala et al., 2001), marketing (e.g., advertising placement, webpages, product label design; e.g., Goldberg et al., 2002, Pieters, 2008, Pieters and Wedel, 2004, Poole and Ball, 2006), psychology (e.g., reading, scene perception, visual search; e.g., Rayner, 1998, Rayner, 2009, Recarte and Nunes, 2000) and landscape perception and design (Dupont et al., 2013, Duchowski, 2007). However, gaze behavior does not provide feedback about why DSS users focus on specific information. In other words, ET cannot be used to determine whether the visually perceived information is relevant for reasoning or decision-making. However, a combination of ET and cognitive interviewing enables an investigation of usability of provided information. To understand this interaction between the use of information integrated in a DSS and cognitive processes, cognitive interviewing, which has been designed to capture cognitive processes and is supported by a large body of methodological research, must be applied (Campanelli, 1997, Campanelli et al., 1991, De Maio and Rothgeb, 1996, Dippo, 1989, Esposito and Hess, 1992, Jabine et al., 1984, Jobe and Mingay, 1991, Jobe et al., 1993, Lessler and Sirken, 1985, Royston et al., 1986, Sirken et al., 1999, Willis et al., 1999, Willis and Schechter, 1997). Thus, by linking information provided by ET with knowledge of cognitive processes, users’ manner of perceiving information provided by a DSS can be investigated.

With this study, we shed light on the usability of an ES-based DSS. We investigated what information is used and how this information affects users’ cognitive processes and reasoning in decision-making. We designed a DSS that displayed ES information in various types of representation. Further, we developed an experimental design based on functions in application of ES information (Klein et al., 2015) to identify key types of representation for communicating ES information. Various functions in application and various experimental tasks were defined that prompted users to apply the information integrated in the DSS in various contexts. These functions in application describe the differences between the purposes the users were applying ES information. Klein et al.'s (2015) results showed that users demand or prefer specific display types or types of representation depending on the intention for using the ES information and thus on the specific function in the application. In the present study, during the experimental runs, the users’ gaze behavior was measured with eye tracking to identify which ES information was used. We also developed a set of cognitive probing questions to investigate users’ reasoning based on perceived information. Finally, to understand how users’ attachment to a location influences their use of information, reasoning and decision strategy, we applied a split-sample design that separated users with and without connections to the region.