“Are You Planning to Follow Your Route?” The Effect of Route Exchange on Decision Making, Trust, and Safety

Abstract

The Sea Traffic Management (STM) Validation project is a European based initiative which focuses on connecting and updating the maritime world in real time, with efficient information exchange. The purpose of this paper is to evaluate two functions developed during the project: a ship to ship route exchange (S2SREX) function and rendezvous (RDV) information layer, collectively referred to as S2SREX/RDV. S2SREX displays the route segment consisting of the next seven waypoints of the monitored route of a collaborating ship and the RDV layer that predicts a meeting point. S2SREX/RDV provides supplementary information to data acquired by existing navigation systems and is intended to improve situational awareness and safety through a more comprehensive understanding of the surrounding traffic. Chalmers University of Technology and Solent University completed an experiment using twenty-four experienced navigators in bridge simulators. Six traffic scenarios were developed by subject matter experts and tested with and without S2SREX/RDV functionalities. Qualitative data were collected using post-test questionnaires and group debriefs to evaluate the participants’ perceptions of S2SREX/RDV in the various traffic scenarios, and quantitative data were collected to assess the ship distances and behavior in relation to the International Regulations for Preventing Collisions at Sea (COLREGs). The results revealed that participants generally trusted the S2SREX/RDV information, and most used S2SREX/RDV for decision support. The quantitative assessment revealed that the COLREGs were breached more often when S2SREX/RDV was used. Experimental findings are discussed in relation to safety, trust, reliance, situational awareness, and human-automation interaction constructs.

1. Introduction

1.1. Background

Technologies emerging within the maritime industry have the opportunity to change shipping as we know it [1,2]. According to Kitack Lim, secretary-general at the International Maritime Organization (IMO), the maritime industry over the next 10 to 20 years will see as much change as we have experienced over the past 100 years [2]. Although this is an exciting time for the maritime industry, it is also a challenging one because the “consequences of change are profound in hindsight, but difficult to anticipate” [3], and how we handle this tran April sition towards “smart ships,” advanced technology integration, and more autonomous systems is of upmost importance. There are endless opportunities for information systems, new technologies, and automation to result in significant performance gains, and the promises of artificial intelligence, big data, and robotics could revolutionize shipping today [4]. In an industry that transports approximately 80–90% of the world’s goods, there is much room for improvement [2]. The fourth industrial revolution is upon us, and there is need to accept the ever-growing plethora of technologies and big data to create a safer and more efficient maritime transport system. However, in order to achieve a safer and more efficient maritime sector during this transition period, a human-tech approach must be adopted [5]. Within the realm of technology acceptance, it is critical to ensure that the human-automation interactions are understood and prioritized so that technologies are properly tested and introduced safely, slowly, and effectively. Otherwise, as we have learned from other industries, accidents can and will happen [6,7,8,9,10].

This rapid advance of technology and digitalization within the shipping industry has been addressed through the IMO’s e-navigation initiative, which is defined as “the harmonized collection, integration, exchange, presentation and analysis of marine information on board and ashore by electronic means to enhance berth to berth navigation and related services for safety and security at sea and protection of the marine environment” [11]. A core aspect of this initiative is the recognition of user needs and a human-centered approach to technology implementation. The Sea Traffic Management (STM) Validation project is a European based initiative which focuses on implementing digital information exchange services between ships and from ships to ports (vessel traffic services) [12]. The STM concept was developed within a previous European Union (EU) project called MONALISA2.0, which provided, among other things, a common route-exchange format and assisted in ensuring the interoperability between equipment manufacturers [12]. These projects and many more (i.e., MONALISA (2010-EU-21109-S), MONALISA2.0 (2012-EU-21007-S), EfficienSea2 (H2020-EU.3.4), and STM (2014-EU-TM-0206-S) were motivated by IMO’s e-navigation strategy [11].

The STM concept promotes a need-to-share attitude instead of need-to-know approach, as is mostly practiced in the maritime industry today. Such a maritime system-wide information management concept would more closely mirror the air traffic control (ATC) model of sharing voyage plans and time stamps among parties both in planning and active traffic situations. Although there are major legislative differences between ATC and vessel traffic services (VTS)—the apparent counterpart in maritime traffic monitoring—they are frequently compared with each other, yet very little cross-domain research exists [13,14]. The STM project generated several information-sharing services, including the Nordic pilot route service; the Baltic Navigational Warning Service; the SSPA Route Optimization Service; the SMHI Route Estimated Time of Arrival (ETA) Forecasts; the winter navigation exchange; and search and rescue, enhanced monitoring service, and ship-to-shore tools. Also included, and the focuses of the research of this paper, are the ship-to-ship route exchange (S2SREX) and the rendezvous (RDV) information layer; they are collectively referred to as S2SREX/RDV [12] S2SREX can be considered an embedded function in the electronic chart display and information system (ECDIS), and RDV is an additional calculated layer predicting the meeting point between two vessels. The STM functions have the ability to integrate information to optimize voyage plans, calculated with real-time data, such as information from meteorological information/warnings providers, ports, particularly sensitive sea areas (PSSAs), and maritime safety information (MSI). The routes can be shared amongst other ships to increase the distributed situational awareness in order to improve safety and efficiency at sea.

As conceptualized in MONALISA2.0 and advanced in the STM Validation Project, the European Maritime Simulator Network (EMSN) was developed. The EMSN consists of 12 connected ship handling simulators based in seven EU countries with technologies to conduct scenarios with over 30 manned simulated vessels. The EMSN is a unique testbed that enables the introduction and testing of new technologies in complex and large-scale traffic situations, without exposing seafarers or the environment to any risks. The EMSN was used for extensive data collection efforts during 2017 and 2018; it assessed several of the STM’s functions at full scale with 512 professional mariners in both baseline (no STM) and STM scenarios [12,15,16,17]. The International Chamber of Shipping (ICS) expressed concern to the STM Validation project management that S2SREX/RDV may encourage the use of the STM compliant ECDISs alone for navigation and collision avoidance decision-making.

Navigational and collision avoidance decisions are based on actions such as looking out the window/keeping watch, and utilizing vessel tracking information (e.g., automatic identification system (AIS), ECDIS, marine radar, and automatic radar plotting aid (ARPA)). IMO Resolution A.1106 (29) discusses the potential of AIS as an aid for anti-collision device. It states that AIS may be recommended as such a device, as long as the information is only used to assist in collision avoidance decision-making with the following cautionary points in mind: (1) AIS is an additional source of navigational information. It does not replace, but supports navigational systems, such as radar target-tracking and VTS, and (2) the use of AIS does not negate the responsibility of the officer of the watch (OOW) to comply at all times with the collision regulations, particularly when determining whether risk of a collision exists [18]. The ICS has concerns that assumptions about approaching traffic might be made based on the projected route on the ECDIS rather than traditional means of collision avoidance, which could be detrimental to safety if there is an overreliance on this information.

These concerns precipitated the present study which focuses specifically on S2SREX and RDV to assess the potential impacts on decision making of the route sharing functions. S2SREXprovides the navigator with a route segment consisting of the next seven waypoints of the monitored route of another vessel. Route segments are broadcasted through AIS, and thus give additional information to the presently available data obtained by radar/ARPA and AIS. Further, targets broadcasting their routes may be acquired to obtain the rendezvous (RDV) information which graphically and numerically presents a prediction of the own ship and STM AIS target’s positions and time of the route based closest point of approach (CPA) based on the actual AIS speed information (Figure 1). On the ECDIS, these points are “clickable,” showing predicted time to go, RDV meeting time, and route CPA. It is important to note that this information is based on the actual monitored route (for both the own ship and the target ship) and broadcasted AIS speed. Therefore, if a vessel deviates from, or does not closely follow its monitored route, the information obtained from S2SREX/RDV will not be accurate. Changes in course (approved or not), must manually update their monitored route and re-broadcast it, so that it is accurate for the surrounding traffic. If the navigators are not fully aware of the data sources for S2SREX and the basis of the RDV calculations, this may lead to cases of over-trust and overreliance on the information presented on the ECDIS. However, the intention behind the S2SREX information was neither to replace the application of the International Regulations for Preventing Collisions at Sea (COLREGs) nor to use it in close-quarter situations. Instead, it was developed as a strategic tool for decision-support and improved situational awareness at longer ship to ship distances.

Traditionally, the maritime industry has lagged behind other transport sectors in terms of adopting and implementing new technologies, and this is largely because of regulatory limitations [19,20,21]. Unfortunately, the traditional culture of shipping is likely the cause for this lag in regulatory change, and has the potential to hinder the acceptance and exploitation of the benefits of automation and digitalization to the fullest extent [19]. Further, the projects and initiatives that have assessed various types of technology and automation within the maritime domain have had a primarily technology-centric approach to testing and implementation [22,23,24,25]. The approach, which largely disregarded the precepts of human-centered design likely led to the slow adoption of technology integration and acceptance. Technology that is being developed to support human decision making should be designed based on the human needs and grounded in the “work as done” concept instead of the “work as imagined” one [26,27]. This situation has led stakeholder organizations (e.g., ICS) to express valid concerns about the proposed technologies and their uses in practice.

1.2. Human-Automation Research

Human-computer interaction (HCI) or human-automation interaction (HAI) research within safety-critical domains (medicine, nuclear, transportation, etc.) increased dramatically in the 1990s and early 2000s [28,29,30]. Parasuraman and Riley (1997) wrote a comprehensive review paper on human interactions with automation which discusses why automation often fails to perform as expected [7,29]. Although this paper was written over two decades ago, the challenges identified related to automation use, misuse, and abuse, are more relevant than ever today in a technology-centered shipping industry [7,29]. Some of the human-automation challenges identified include trust, overreliance, mental workload, risk, situational awareness (SA), and automation biases, some of which are discussed in the current paper [7,29,31]. During the transition towards smart ships, the partnership between humans and machines is of utmost importance for working together to achieve a desired result (i.e., safe navigation). Many industries have addressed the human-automation requirements with the adoption of a levels of automation (LOA) scale usually ranging anywhere from 0 to 10, with each level representing the human involvement from 0 = no automation to 10 = fully autonomous and everything in between [31,32,33,34]. Although the shipping industry has been slow to adopt the LOA regime, the International Maritime Organization (IMO) recently addressed degrees of autonomy of maritime autonomous surface ships (MASS) with a four level scale [35]. The present paper studies the effects of low-level automation which would reflect the lowest level of autonomy defined by the IMO: a ship with automated processes and decision support; “Seafarers are on board to operate and control shipboard systems and functions. Some operations may be automated” [35]. This type of human-automation relationship requires even more careful consideration as to how and which tasks become automated and how this will impact the SA and decision-making processes of bridge personnel.

Another issue which has dominated human-automation research considers human trust of automation. Trust is a construct in which a human considers the reliability, truth, or ability of a system and is related to the concerns regarding the misuse and disuse of technology [7]. Over-trust leads to misuse, which is when operators rely on automation when the automation performs poorly, and under-trust leads to disuse, which is when operators fail to engage automation when it could enhance performance [7,8,9]. Within the transportation sector there are countless cases in which people have either misused, disused, or abused automation which has led to major accidents: notably, the Royal Majesty cruise ship [14,36], and too many aviation incidents [29,37]. On the other hand, there is a certain level of skepticism in trusting automation, particularly in industries which have not traditionally been highly automated (i.e., shipping) which can undermine the benefits of automation [8,38]. Even though there have been major developments in on-board ship technology, the work practices and procedures reflect a more traditional workplace [19].

One of the most highly debated topics within human factors is situational awareness, yet this term is used in all forums, both technical and non-technical [39,40,41]. As defined by Endsley, SA is, “The perception of the elements in the environment within a volume of time and space, the comprehension of their meaning and the projection of their status in the near future” [42]. However, the definition of SA remains a topic of contention amongst academics, and even more disagreement surrounds how it should be accurately measured [43]. Depending on which approach one chooses to define and evaluate SA, there are three common view-points: psychological, engineering, and systems [39]. The psychological approach is the most common and widely cited, which uses Endsley’s three level perception-comprehension-projection model [44]. The engineering approach is commonly applied within military and transportation research. This is a technology-focused approach, meaning that displays, sensors, maps, etc., have SA (i.e., a navigation display contains SA for a pilot) [45,46]. More recently, a holistic systems approach to SA has been discussed, which originated from the distributed cognition movement of Hutchins in 1995 [39,41,43,47,48,49]. The term distributed situational awareness (DSA) is used within complex socio-technical systems to describe how people work together, and how information bonds people and technology together [39,47,48]. Although the authors of this paper believe in a more holistic systems-based approach to SA, because of time and monetary constraints associated with the considerable intensive analysis of systems SA, the analysis, methodology, and use of the term SA in this paper reflect predominantly an engineering-based approach to SA.

1.3. Purpose and Research Questions

S2SREX/RDV introduces low level automation, designed to support human decision making in navigation. At this stage of development, it is unknown whether and if so how these functions will affect the behaviors and the decision making of the OOW in traffic situations. The ICS has concerns about how this information will be used and whether it will negatively influence current collision avoidance practices. The purpose of this study was to evaluate S2SREX/RDV in a controlled simulator setting to assess how it may affect the decisions and actions taken by navigators in various traffic situations. The following research questions were addressed:

• Does S2SREX/RDV influence decision making?
• Do the participants trust S2SREX/RDV information?
• Does S2SREX/RDV make navigation safer? (Considering the risks of misuse and overreliance on the information.)

2. Methodology

2.1. Participants

Participants were recruited to participate in simulation trials at either Chalmers University of Technology (CTH) in Gothenburg, Sweden, or Warsash School of Maritime Science and Engineering Southampton Solent University (WMA), in Southampton, UK. Selective recruitment was used to ensure the participants were familiar enough with both the simulator equipment and the STM functions installed on the ECDIS (i.e., STM Client). The following participant selection criteria were applied: participants were either required to have participated in the STM EMSN simulations (March 2018 or June 2018) fulfilling the minimum qualification requirements stated for these trials, i.e., professional mariners (active or recently active masters, mates, officers, and maritime pilots); or new participants had to be professional mariners (active or recently active masters, mates, officers, and maritime pilots) familiar with the simulator equipment and available to attend a familiarization session prior to the data collection period.

2.2. Ethics

Participants were fully informed of the procedures and risks of the experiment and signed electronic and written informed consent prior to the start of the simulations. The experiment complies with the requirements of article 28 of the EU General Data Protection Regulation (2016/679) regarding protection for physical persons in the processing of personal data. Each participant was assigned a unique identification number (ID) prior to arrival, which was used for all the questionnaires throughout the study to maintain confidentiality.

2.3. Demographics

A total of twenty-four test participants (three females and twenty-one males) were included in this study, twelve at CTH and twelve at WMA. Nineteen participants were from either Sweden or the United Kingdom, and the others were from Latvia and Nigeria. The majority of the participants were between 30 and 49 years old, the rest of the participants were 20–29 or 50–59. There were ten deck officers, six chief officers, three captains, and one VTS operator among the test participants, and the remainder selected “other” in the questionnaire. In terms of computer literacy, approximately 50% of the participants started using computers between 6 and 11 years old, and approximately 20% of participants did so between 12 and 15 years old. The majority of participants (75%) indicated that they spent more than 31 h per week on a computer.

2.4. Technical Set Up

Full mission bridge simulators with visual surveillance, CCTV, and audio channels were used to collect the data. The bridge layout and equipment differed slightly between WMA and CTH, but generally consisted of a modern cockpit design with real ship bridge consoles, visual channels, and monitoring and control equipment. Each bridge was equipped with ECDIS and radar, a conning display, GMDSS, and communication consoles using VHF and intercoms. For the scenarios using S2SREX and RDV, the STM client using Transas MNS35 (with incorporated STM functionalities) software was integrated and replaced the standard ECDIS.

A series of traffic scenarios were developed by simulator instructors (SME´s) for full-scale testing in the simulators. The simulation scenarios covered a variety of traffic situations, traffic separation schemes (TSS), including longer and shorter-range scales in more confined waters. In this study no scenarios were set in VTS areas, port areas, and/or areas with compulsory pilotage. Three vessel types of various sizes were used in the traffic scenarios, set in calm weather conditions and good visibility. The scenarios were approximately thirty minutes long and provided enough time for the test participants to assess the traffic situation and take action if needed. The scenarios (1–6) are described in

Table 1. Description of simulation scenarios (lines indicating the monitored routes).

1	Overtaking and crossing situation in an area with recommended routes and TSS (Hatter Barn TSS).	In areas with recommended routes and traffic junctions where ships may take different routes, S2SREX may support decision making by providing information on other ships routes and thereby predict and identify possible close quarters situations.
2	Potential meeting in a bend (Hatter Barn TSS).	S2SREX may support decision making regarding knowing which routes the vessels involved take, if there will be a meeting, where the meeting point will be and result in an adaption of speed to meet at a safe location.
3	Multiple crossings with close quarters situations if all vessels follow their routes (South of Isle of Wight).	Using ARPA only, no close quarters situations are apparent. However, using S2SREX will show that all 3 ships will meet at the same point.
4	Overtaking and being overtaken in a TSS/ Precautionary Area (Bornholm Gatt).	May support decision making regarding if and on which side to overtake in a TSS when approaching a Precautionary Area by being able to determine in advance other ship’s intended routes once they enter the Precautionary Area.
5	Meeting and crossing (The Skaw).	May support decision making in a traffic area where it could be helpful knowing when a ship is intending to change course as to round the Skaw.
6	Crossing, meeting (Dover Strait).	Decisions based on S2SREX and RDV may suggest that if all ships follow their routes and speeds, no close quarters situation will develop. However, this may be regarded as a breach of the COLREGs.

.

2.5. Experimental Set-Up

A one-day pilot study was completed at CTH and WMA prior to the official data collection with three in-house, highly experienced mariners. Data were collected over four days in October 2018 at each simulation center, and the same protocol was followed by CTH and WMA for all aspects of experimental set-up and data collection. Each scenario (S1–S6) was run every day as either a control/base line condition or an experimental condition (S2SREX/RDV). The order of the scenarios was randomized to avoid any learning effect [50]. There were three participants per day (3 participants × 4 days) at each simulation center (12 participants at each facility), rendering a total number of 24 participants. For each scenario, a baseline and S2SREX/RDV were tested by twelve different test participants acting as the OOWs for their respective vessels involved. The only difference between baseline and the experimental condition was that the test participants had access to an ECDIS with S2SREX and RDV functionality. In all scenarios, the routes were pre-planned by the instructor and set on monitoring on the ECDIS before the start of the exercise.

2.6. Data Collection

A simulator instructor and a human factors specialist observed the trials from the control room of the simulation center. Each morning, all participants were given identical briefings at each simulation center, followed by a familiarization session with the S2SREX/RDV functions. The participants were given a chance to test the functions in several exercises predefined by the simulator instructor. After each simulation scenario (S1–S6), each participant filled in a brief post-scenario questionnaire regarding his/her perceived performance and opinions about the scenario and STM functions. At the end of each day, there was also a common open-ended debrief which included semi-structured interviews to obtain information related to the participant’s overall perceived performance and opinions about S2SREX/RDV [51]. During the debrief, the scenarios were replayed to help the participants remember what happened in each scenario and discuss the outcomes. The purpose of this exercise was to probe the participants to think about how S2SREX/RDV influenced their decision-making processes and how it could impact safe navigation practices.

2.7. Data Analysis

The questionnaires were developed using online survey software, Qualtrics (QualtricsXM, © 2019, Provo, UT, USA, https://www.qualtrics.com), and the questionnaires consisted primarily of Likert scale questions and several optional free-text sections. The questionnaire data were analyzed using basic descriptive statistics in excel and are summarized in Section 3. A lightweight qualitative data analysis approach was used to analyze the debrief sessions [52]. This data analysis consisted of reviewing the field notes, which were taken during the debriefing sessions, accompanied by parallel discussions with subject matter experts and simulator instructors to understand and draw conclusions from the data. The videos were mainly used for verification or clarification purposes (instead of for transcription). The numerical analysis was completed by replaying the log files of each simulation run on a simulator instructor station, and the following values were noted: the ships in the traffic situation; the type of situation (crossing, meeting, or overtaking); identification as to which ship was standing or giving way; the indicated closest point of approach (CPA) and time to closest approach (TCPA) before taking action (either a change of course or speed); the resulting CPA after the action was taken; whether the action constituted a breach of the COLREGs; and any additional comments. Concerning the action(s) taken, each action was recorded as an event; i.e., a ship could perform several actions during the time of the scenario consisting of several course changes and/or speed changes. Course changes of less than 4 degrees were not noted, as it was not verifiable whether the course change was meant to be an avoiding maneuver. CPA and TCPA values were taken from the calculations from the simulator software (instructor station) and may be slightly different from the values presented on the automatic radar plotting aid (ARPA). It is commonly debated amongst experts how to define a “Breach of COLREGs” and opinions may vary based on what and when something is considered a breach. In this analysis, a breach was conservatively defined as, “An action against the COLREGs at a distance of less than or equal to 3,5 Nautical Miles (NM) and TCPA less than 12 minutes regardless of whether the ships involved in the scenario agreed on the avoiding maneuver by e.g., VHF.”

3. Results

3.1. Post-Scenario Questionnaire

The post scenario questionnaires yielded 143 valid responses and one corrupted response (three responses per scenario × six scenarios per day × eight days). Seventy-two questionnaires were from the baseline scenarios and 71 from the S2SREX simulations. All (100%) of the participants chose to use S2SREX when it was available. The top three reasons selected by the participants to use S2SREX were: “To enhance situational awareness”; “To supplement information from other means (ARPA, AIS, etc.)”; and “To help in assessing whether a close-quarters situation was developing.” The S2SREX/RDV function combination was used less than the standalone S2SREX function; 74.65% of participants reported using this function, and the primary reasons cited by participants for not using RDV were: it was not considered helpful (76.19%), or unnecessary (14.29%) in that particular situation. (e.g., “CPA through radar was enough in this situation,” “no need,” and “in this situation it made no difference. Seeing the routes of the other vessels was enough both to make decisions and to understand the other vessels’ intentions.”) The RDV information layer was used primarily in assessments of developing close-quarters situations.

In terms of SA, 95.8% of participants believed that S2SREX improved SA, and 67.61% of the participants indicated that they made a decision based on S2SREX information (

Table 2. Frequency distribution of S2SREX and S2SREX/RDV for situation awareness (SA), decision making, and clarity of the information.

S2SREX	N	Yes	No
Did S2SREX improve your SA?	71	68 (95.8%)	3 (4.2%)
Did you make a decision based on S2SREX information?	71	48 (67.6%)	23 (32.4%)
Were you confused about the information displayed?	71	7 (9.9%)	64 (90.1%)
S2SRX/RDV
Did S2SREX/RDV improve your SA?	53	45 (84.9%)	8 (15.1%)
Did you make a decision based on S2SREX/RDV information?	53	28 (52.8%)	25 (47.2%)
Were you confused about the information displayed?	53	6 (11.3%)	47 (88.7%)

). Less participants used the S2SREX/RDV function, but out of 53 respondents, 84.9% believed that S2SREX/RDV improved their SA (Table 2), and 52.83% responded that they made a decision based on S2SREX/RDV information. In general, participants did not seem confused by the information displayed from both functions.

In terms of perceived importance of the functions on decision making, 41.7% of responses suggested that the S2SREX was considered “very important” in the decision-making process, while 46.4% thought that was also applicable to the RDV information layer (Figure 2).

3.2. End of Day Questionnaires and Semi-Structured Interviews

In addition to the post-scenario questionnaires after each scenario (S1–S6), there was an additional questionnaire which asked about decision making, trust, and overreliance for the S2SREX/RDV functions. These results include 24 responses from all 24 participants from both CTH and WMA. The results are presented in

Table 3. Summary of end of the day questionnaire results.

Navigational Tendencies	n*	Extremely Unlikely	Somewhat Unlikely	Neither Likely nor Unlikely	Somewhat Likely	Extremely Likely
Knowing the monitored route is broadcasted, do navigators follow their routes to a higher extent? (i.e., less willing to deviate from their route?)	24	1 (4.2%)	4 (16.7%)	10 (41.7%)	9 (37.5%)	0 (0%)
Tendency for a shift towards using the ECDIS (with S2SREX and RDV information) instead of ARPA/visual means when ascertaining the risk of collision?	24	0 (0%)	5 (20.8%)	2 (8.3%)	16 (66.7%)	1 (4.2%)
Trust		Never	Sometimes	About Half of the Time	Most of the Time	Always
Do you consider S2SREX information as trustworthy?	24	0 (0%)	4 (16.6%)	3 (12.5%)	16 (66.7%)	1 (4.2%)
Risk and Overreliance		No Risk	Low Risk	Medium Risk	High Risk	Extremely High Risk
Is there a risk that navigators put overreliance in S2SREX?	24	0 (0%)	2 (8.3%)	8 (33.3%)	12 (50%)	2 (8.3%)
Is there a risk for misinterpreting data obtained from S2SREX and RDV?	24	0 (0%)	3 (12.5%)	12 (50%)	8 (33.3%)	1 (4.2%)

n* = number of respondents.

.

The results from the semi-structured interviews during the debrief generally supported the questionnaire results. The major findings from the light qualitative data analysis include:

• Positive impact on navigational safety: The participants shared a positive attitude towards the usefulness of the functions and perceived that S2SREX/RDV will improve SA. They also believe that the functions will increase the available time to respond to potentially dangerous situations, and they placed a high level of trust in the information.
• Purpose of STM functions: There was discussion surrounding the purpose of the functions; i.e., when should they be used (strategic long-term planning) and when they should not be used (tactical tool for collision avoidance). The participants claimed to be aware of the potential risks and challenges associated with the functions (i.e., violation of COLREGs, information overload, decisions made based on assumptions, etc.). These findings are further discussed in Section 5.
• Usability: This was the only result which did not directly align with the questionnaire results. The participants mentioned several issues related to usability, including: the overlapping routes of other vessels on the ECDIS, the fact that all the routes have the same color, and some difficulty with the RDV information layer. However, the participants discussed that usability would be more of an issue in traffic scenarios with more than three vessels.

3.3. Results from Numerical Analysis

Initially it was considered that we could evaluate each scenario (described in Table 1) separately; however, the amount of data was not sufficient to permit a scenario-based analysis, and it was decided to cluster the scenarios into two groups:

• Scenarios 1, 2, and 4: Meeting and overtaking scenarios in confined waters (recommended routes, TSS) where planning for meeting/overtaking may be important.
• Scenarios 3, 5, and 6: Crossing scenarios in more open waters.

The results are presented in

Table 4. Numerical analysis of baseline and experimental conditions.

All Scenarios	Distance When Taking Action (NM)*	Resulting CPA (NM)*	Breach of COLREG
No S2SREX	3.6	0.9	2
with S2SREX	4.1	1.1	11
Means in Meeting/Overtaking Scenarios 1, 2 and 4
No S2SREX	2.4	0.7	0
with S2SREX	2.6	0.9	3
Means in Crossing Scenarios 3,5 and 6
No S2SREX	4.4	1.1	2
with S2SREX	5.2	1.3	8

* NM is nautical miles.

.

4. Limitations

This study was limited to six traffic scenarios and could yield different results in different scenarios or simulated conditions. The selected scenarios were developed based on the assumption that S2SREX/RDV might have an influence on decision making. All scenarios were completed in ideal conditions: daylight, calm weather, and good visibility, and there are many additional factors (i.e., geographical area, type and condition of vessel, etc.) which could possibly affect the decisions and behaviors of navigators in similar traffic situations. The participants in this study were predominantly from Sweden and the UK; thus, the generalizability of the results may be limited to nautical personnel in these geographical areas. In addition, this study is based on the subjective feedback and experiences of the participants collected through post scenario questionnaires. Moreover, this study was completed in a simulator, which might produce different results than in reality; however, given the safety-critical aspects of this study, simulator testing was the most viable option. Finally, the numerical analysis does not include enough data to draw any statistically supported conclusions and therefore must be interpreted cautiously.

5. Discussion

Introducing new technology into a system (i.e., a ship’s bridge) does not simply manipulate a single variable; instead, it changes the entire socio-technical system; transforming judgements, roles, relationships, and weightings on different goals [3]. Few technologies are instantly accepted when introduced to a workplace; it takes time and experience to learn to trust and rely on them [29]. Although the technology introduced within the present study represents a relatively small system change, altering the means of information retrieval will impact information processing, perception, and decision making [44]. Parasuraman and Sheridan (2000) identified a four-stage model of human information processing and automation capabilities: (1) information acquisition; (2) information analysis; (3) decision and action selection; (4) action implementation [31,53]. The authors indicate that the first two levels can be grouped together as “information automation,” which matches the level of automation proposed by S2SREX/RDV [31]. S2SREX/RDV provides relevant information about the intended routes and predicted RDV points of surrounding traffic in an automated function. It is important to note that these functions do not replace the human action of decision selection and implementation or overrule the COLREGs; they are exclusively intended to be used for decision support at longer ranges.

5.1. Decision Making and Trust

The most interesting findings in this study are that almost 70% of the participants (Table 2) believe that S2SREX influenced their decision-making, while approximately 50% of participants believed this was the case for the S2SREX/RDV function (Table 2). Further, almost half of the participants suggested that S2SREX was considered “very important” in the decision-making process; there were similar results for the RDV information layer (Figure 2). Therefore, from the participants’ perspective, S2SREX/RDV strongly influenced decision making. In addition, the participants placed a high level of trust in the S2SREX information, as almost 70% of participants said they would trust the function most of the time, and no respondents said they would never trust the S2SREX information (Table 3). These results are optimistic, given the relatively limited amount of familiarization time and the fact that the participants were particularly aware of the possible conflicts of using S2SREX in various situations.

The perceived level of trust in technology is one of the primary concerns in the human-automation relationship [8]. Over-trust leads to misuse, which is when operators rely on automation when the automation performs poorly, and under trust leads to disuse, which is when operators fail to engage automation when it could enhance performance [7,8,9]. Finding the optimal level of trust in new technologies is difficult because trust is complex and can have a fundamental impact on human-automation interactions, safety, and system performance [8]. John Lee and colleagues have widely published research on trust, primarily in vehicle automation, but also in other highly complex human-machine systems [8,36,54]. One of the many concepts they discuss is related to self-confidence and how high or low self-confidence serves as a major predictor related to the use and reliance on automation [8]. They also discussed the idea of “exploratory behavior,” which is a contributing factor to reliance and trust and is more than likely applicable to the S2SREX/RDV functions. Exploratory behavior is common in experimental settings in which people are tested in “microworlds,” which are “simplified versions of a real system in which the essential elements are retained and the complexities eliminated to make experimental control possible” (i.e., full-mission simulators) [10,55]. In this case people explore the possibilities of automation and knowingly compromise system performance to learn how it works or behaves, which could influence how it is used [8,10,55].

A person´s level of trust is also related to how information is presented to them [8]. The content, format, interface, and usability have been shown to have powerful impacts on trust, even if such things are not correlated with the true capabilities of the system [8,56]. S2SREX is a newly developed function and has not been subject to sufficient usability testing. User testing and feedback should be incorporated throughout the entire development lifecycle of the functions [22,23,24,57]. Poor usability, and “cluttered screens” caused by overlapping routes of different vessels and difficulties identifying which route belonged to which ship were mentioned by several participants throughout the simulations and during the semi-structured debriefs. These usability issues caused frustration and could impact trust, and therefore proper use of the functions. This result was not reflected in the post-scenario questionnaire results in which only 9.86% of the participants thought that the S2SREX function was confusing, and 11.32% for the S2SREX with RDV information layer (Table 2). However, the scenarios did not simulate busy waterways, as only three ships were involved, limiting the potential clutter on the screens. This was more prevalent when more of the STM functions were tested in the full-scale EMSN simulations with more dense traffic situations [15]. It is suggested that user testing is completed prior to live implementation of the technology in order to better understand the user needs and potentially address any confusion related to the intended use of the technology. This would considerably improve safety and would allow the participants to adopt an appropriate level of trust and reliance on the information provided by the functions [8]. It is also important to recognize that trust is a complex phenomenon and cannot be fully measured using a post-test questionnaire; additional testing using mixed methods is suggested for further studies.

It is difficult to fully assess whether or not these results would translate into real world usage of the functions when safety or system performance could be compromised. The results indicate that the participants appreciate and trust the S2SREX/RDV information and strongly believe it influences their decision making. Although this is a very positive result for the STM Validation project and user acceptance of the technology, it is important to acknowledge the potential for over-trusting the information, given the maturity of the technology.

5.2. ICS CONCERNS

STM provides a new platform on which to exchange information between ships, which may alter current navigation practices. S2SREX/RDV broadcast information that was not previously available to the navigators, and is intended to provide a more complete distributed awareness of the surrounding traffic. These functions are intended to be used during longer range (strategic) navigation situations and not during tactical (close range) navigation. The ICS has raised concerns about how this information will impact safe navigation practices. In particular, the concerns relate to navigators’ potential assumptions in decision making if the traffic situation and risk of collision is assessed based on the broadcasted routes (as presented on an ECDIS with S2SREX/RDV functionality) rather than by current navigation practices, such as visual and radar assessment of the range, relative bearing, movement, and aspect. In addition, one of the notable issues with the S2SREX/RDV function is that the information presented in the ECDIS is based on the actual monitored route (for both one’s own ship and the target ship) and broadcasted AIS speed. Therefore, if a vessel deviates from or does not closely follow its monitored route, the information obtained by S2SREX and RDV will not be accurate. The navigators would need to actively update (i.e., re-plan and re-monitor) their routes for this information to be correct, which is hardly conceivable in a typical traffic situation. Therefore, the technology as is, creates the potential to cause confusion and potentially dangerous situations between vessels, unless navigators are fully aware of the data sources and reliability of S2SREX and the basis of the RDV calculations.

The questionnaire results indicate that most of the participants (66.7%) agreed that there was a “somewhat likely” chance of a shift towards using the ECDIS (with S2SREX and RDV information) instead of radar/ARPA/visual tools, while another 20% of the participants believed that was “somewhat unlikely” to happen (Table 3). This result supports the ICS concerns, meaning the participants generally agree that they will shift their primary information source to S2SREX/RDV when assessing traffic and the risk of collision. During the semi-structured interviews, the participants agreed that the functions should only be used as intended (i.e., longer-range strategical navigation) and expressed concerns and uncertainty about S2SREX/RDV being mis-used, which could present a potential liability and safety concern. An additional issue is related to the shared information provided to the surrounding traffic, and whether navigators would follow their routes to a higher extent (i.e., be less willing to deviate from their broadcasted route). The results were primarily neutral for this question, as 37.5% of participants thought this was somewhat likely to happen and 42% selected neither likely nor unlikely (Table 3). This question might be difficult to answer given the short duration of the scenarios; further qualitative assessment is recommended.

Based on the limitations of S2SREX/RDV, there is also the risk of overreliance and misinterpretation of the data. The participants generally trusted the S2SREX/RDV information, yet 50% of the participants assessed the risk of overreliance on S2SREX information as high, and 8% rated the risk as extremely high (Table 3). These findings indicate that the participants are acutely aware of the inherent risks in using the functions, yet they heavily relied on this information for decision making. Interestingly, the risk of misinterpretation of the data was rated as less likely (Table 3). These findings also align with previous maritime human-automation research, which encourages the industry to “proceed with caution” when introducing new technologies within complex socio-technical systems [58]. There must be further testing related to data and system integrity, redundancy, and continuity to meet the e-navigation objectives, while also meeting user needs and expectations [11]. A balance is needed between realizing the benefits of automation, while maintaining an appropriate awareness of its capabilities, limitations, and potential impacts upon decision making.

5.3. Safety

Improving navigational safety is one of the primary aims of the STM project, with an ambitious goal to reduce maritime accidents by 50% through the year 2030 [59]. Many factors can influence navigational safety, and this study assessed a relatively small change to navigation practices within a much larger socio-technical system. The participants’ perceived safety was assessed through a post-trial subjective rating of individual situational awareness and semi-structured group interviews. These assessment methods provide insight into the participants’ understanding or perception of safety, which represent “work as imagined.” The participants believe that S2SREX/RDV information highly contributed to enhanced perceived SA (Table 2) [44,45,60]. This positive attitude was echoed throughout the semi-structured interviews; participants generally believed that SA and safety would be improved with S2SREX/RDV.

SA is tightly coupled to safety, and a loss of SA is correlated with poor system performance [61]. Therefore, it is critical that proposed technologies enhance and do not detract from the operators’ SA. Endsley (1995) proposed eight design criteria guidelines for enhancing SA which align with the intent of the S2SREX/RDV functions: (1) Reduce the requirement for people to make calculations. (2) Present data in a manner that makes level 2 SA (understanding) and level 3 SA (prediction) easier. (3) Organize information in a manner that is consistent with the persons goals. (4) Indicators of the current mode or status of the system can help the cue the appropriate situational awareness. (5) Critical cues should be provided to capture attention during critical events. (6) Global situational awareness is supported by providing an overview of the situation across the goals of the operator. (7) System-generated support for projection of future events and states will support level 3 SA. (8) System design should be multi-modal and present data from different sources together rather than sequentially, in order to support parallel processing of information [44]. The functions tested in this study generally meet Endsley’s design criteria, and if the functions are used as intended (i.e., for strategic planning), S2SREX/RDV should improve navigational safety. However, the fifth design criterion—“critical cues should be provided to capture attention during critical events”—should be further developed for the S2SREX/RDV functionality. If possible, it should be more obvious that a ship is not following its intended route to avoid any confusion and decrease the risk of misinterpretation. Assessing SA from a post-trial subjective questionnaire does not provide the necessary information to make a statement about navigational safety. Additional means of SA assessment are needed to determine the full effects of S2SREX/RDV on the entire bridge team or distributed SA within the bridge and shore-based operators.

Another aspect of safety is the “work as done”—or the actual behavior of the navigators throughout the scenarios. This included the numerical assessment of the physical distances between ships when taking action, CPA values, and the total number of breached COLREGs. The differences in the mean distances when taking action are not statistically significant (as there was not enough data to perform a statistical analysis). However, the data suggest that there was a tendency for the officers to take action at a slightly longer distance when having S2SREX/RDV available compared to the baseline (Table 4). In addition, the resulting CPAs when using S2SREX are slightly larger than the baseline scenarios (Table 4). However, when S2SREX/RDV was available, the COLREGs were breached eleven times, whereas in the baseline they were breached twice (Table 4). The breaches of COLREGs did not result in any collisions, although this warrants a further discussion related to safety. Do the COLREGs, as they are written today, support decision aid functions? Would route sharing change the way the COLREGs are applied, and what about a mixed system including ships with STM and ships without? Existing research has shown that in addition to the formal COLREGs, there are also informal rule systems that exist independently between vessels [62,63]. The introduction of new decision support tools into a system may complicate or alter the operator’s behaviors. This is an important finding that must be further explored in both simulated and live testbeds.

The scenarios developed in this study provide concrete examples of how S2SREX/RDV could either be useful or cause unnecessary confusion for navigators (Table 1). Scenarios 1, 2, and 4 are examples of how S2SREX/RDV could be helpful in traffic situations (e.g., meeting and overtaking), and where it would be advantageous to know which route a vessel is planning on going to prevent unwanted outcomes. These scenarios represent the “ideal” situations for the use of S2SREX/RDV. Scenarios 3, 5, and 6 present more challenging traffic situations (mostly crossings) with added S2SREX/RDV functionality (Table 1). In these type of traffic situations, the additional information could unintentionally overload the operator and lead to potentially unexpected navigational decisions. Without S2SREX/RDV, the solutions to these traffic situations would be relatively straightforward based on the COLREGs. This situation is reflected in the results, which indicate that eight out of eleven occurrences of the breached COLREGs happened during scenarios 3, 5, and 6 (Table 4). Although the amount of data is not sufficient to draw conclusions, it is an interesting finding that should be further explored to determine the optimal traffic and use cases for S2SREX/RDV, or similar decision support functions.

Another critical aspect of navigational safety is related to the quality of communication between vessels, which today is primarily VHF radio [26,64]. S2SREX/RDV functionality introduces an additional means of indirect communication via information sharing through broadcasted routes. A limitation of the present study is that VHF radio traffic was recorded, but not analyzed according to each traffic scenario. This information could help enlighten the decision-making process and provide insight into the occurrences of breached COLREGs. It would be helpful to know whether verbal agreements were made or whether navigators implicitly made assumptions based on the S2SREX/RDV information. Based on the results from the present study, it is not possible to say how navigational safety will be influenced with the use of S2REX/RDV. The results present a contradiction between “work as imagined” and “work as done”. The participants perceive that S2SREX/RDV will improve their SA, and influence their decision making while maintaining an awareness of the risks—this represents the “work as imagined”. In contrast, the behavior or “work as done” shows that the navigators breached the COLREGs on more occasions when S2SREX/RDV was available. This result encourages the need for more qualitative research to deepen the understanding of how and why the participants choose to use the S2SREX/RDV and potentially violate COLREGs.

6. Conclusions

This study assessed the S2SREX and S2SREX/RDV functions from both a human (cognitive) and numerical (physical ship distances) perspective. The purpose was to determine whether decision making, trust, and navigation behavior were influenced by the functions. The results showed that the participants generally placed a high level of trust in the S2SREX/RDV information and believed it considerably improved their SA and influenced their decision making. However, the participants acknowledged the risks associated with using S2SREX/RDV for navigation, notably overreliance, or misinterpretation of the data (i.e., when the “route” and “intention” are implicitly assumed to be same). The concerns from the ICS were supported based on the participants’ feedback that there would likely be a tendency to use an STM compliant ECDIS for navigation and traffic assessment alone, compared to traditional means. The numerical results revealed that participants took action at slightly longer distances, which resulted in larger CPA values; however, more COLREGS were breached when S2SREX/RDV were used. It appears there is a difference between “work as imagined” and “work as done”, which requires further investigation. Additionally, the usefulness of these functions appears to highly depend on the traffic situations (i.e., dense or light) and the intended use (i.e., strategic or tactical navigation). Given these results, it is impossible to determine the influence of S2SREX/RDV on “safe navigation” practices.

Although some of the risks identified in this study could potentially be mitigated with detailed procedures, training, familiarization, and similar quick-fixes for the operators at the “sharp end,” this will not address the underlying larger safety concerns [65]. It would instead feed into the cycle of failed technology attempts, and “human-error” related incidents within the maritime sector. The maritime industry requires a different kind of thinking, one that adopts a human-centered approach to technology development and implementation and recognizes the capabilities and limitations of human operators. If addressed from this perspective, technology, automation, AI, and other such things have the power to positively support decision making and improve navigational safety. Further research is recommended to test team and distributed situational awareness, mental workload, and trust development over a longer time period of time. Additional qualitative studies are recommended to understand how and why operators use S2SREX/RDV, or similar things, for navigational decisions and whether or not current COLREGs support this. A longitudinal study would be optimal to assess the level of trust over time given familiarization and experience with the S2SREX/RDV in various testing conditions.