1 Introduction and background

Three hundred sixty–degree (360°) video as a form of immersive experience became popular following the release of low-cost VR headsets for consumer purposes that support a wider field of viewing range and stereoscopic display. HMDs allow users through head movement to choose their own field and direction of view (FoV) simulating a real-world viewing experience. Ferrari & Medici [16] have affirmed the benefits of the use of HMDs for communicating cultural information in VR.

The potential for immersive experience provision by 360° videos led YouTube to provide support for such media upload and display in March 2015, through its official website and Android application [30]. A new field of VR experiences was also introduced that of Cinematic Virtual Reality [20]. The difference between Cinematic VR and 360° video applications lies in the better sense of presence, accessibility and camera motion offered by the former due to the use of VR headset devices. Three hundred sixty–degree immersive video allowed viewers to become more active by choosing their own point of view to experience a scene and not just following the director’s frame shot, providing in this way a more personalised and realistic experience [15]. Three hundred sixty–degree video VR storytelling experiences allowed producers to create longer lasting impact to the audience though semi-interactive experiences that fall between game design and film production. This is achieved by making users to feel like taking part in the action and the narrative [28].

The work of Adão et al. [1] introduces a system specification for prototyping immersive experiences based on 360° video complemented with other forms of multimedia content reporting on good levels of functionality-centred usability. The design of 360° immersive video applications is challenging mainly due to the use of 2D video resources for the creation of interactive experiences in 3D VR worlds, the lack of sense of depth and the weak level of navigation control. It is therefore necessary to follow a rigorous methodology to direct the design of such immersive experiences in order to offer satisfying user experience. Such challenges that occur when using 360° video to design interactive and immersive applications for VR headsets have been thoroughly presented by Argyriou et al. [2].

A systematic design study has been formed with the aim to address those challenges. As defined by Pavlik [21]: “Experiential media forms enable the user to experience stories as a participant in a first-person narrative, rather than merely watch, listen or read the story from a third-person voyeuristic vantage point”. Three hundred sixty–degree immersive video storytelling applications represent a form of experiential media when experienced through VR headsets.

The key research objective of the study presented in this paper is twofold:

  • The definition of the design considerations and tasks that should be addressed when creating immersive interactive experiences based on 360° videos;

  • The introduction of a systematic way to address those design considerations through the contribution of a formal methodology and a set of design guidelines.

The paper structure is as follows. Section 2 introduces Immersive Cultural Heritage solutions developed using 360° video resources presenting some open challenges in that area of research. With the aim to address the identified research challenges, an analysis and categorisation of design tasks and corresponding techniques that need to be considered in 360° immersive video experiences is discussed in Section 3. Those design techniques have been followed for the creation of a 360° immersive video storytelling tour in an area of Cultural Heritage (Section 4) and subjected to a “real-user” study (Section 5). The results of this study are presented in Section 6, followed by a contribution of a set of design guidelines for this new kind of experiences (Section 7) and concluding with discussion and future work (Section 8).

2 Current solutions for Immersive Cultural Heritage based on 360° videos

Digital storytelling is proposed as an effective way of introducing cultural heritage information [4]. Recent studies reveal that digital, immersive storytelling based on experiential media, such as 360° videos, is a promising and engaging new form of experiencing Cultural Heritage. The design of applications which conveys cultural information through Mixed and Virtual Reality technologies should assist the interpretation of projected historical artefacts and increase the user’s interest on them and not on the medium [26].

The work of Ivkovic et al. [17] presents an interactive cultural heritage application based on 360° videos for the Bridges of Sarajevo. Participants were engaged to further explore the story of the seven bridges through the ability to choose on their own the order to experience the provided cultural information. The application integrated a map interface menu with 3D models of the Bridges of Sarajevo that formed an element to navigate the environment. Participants were further motivated to explore all the provided content with the integration of a rewarding mechanism, through the collection of puzzle pieces for each visited bridge. The completion of this puzzle allowed participants to explore an extra video scene. This study reports that participants felt like they were realistically walking on the bridges when experiencing the video stories. That application though was web-based, and therefore, there was no discussion on design considerations for VR headset applications, promising for increasing the sense of presence.

Selmanovic et al. [24] introduce an immersive 360° video application for preserving the bridge diving tradition from the Old Bridge in Mostar, Bosnia and Herzegovina. The users watch 360° videos about the bridge’s history and the diving tradition and answer a series of quizzes on the presented information. It is an effective approach of designing immersive cultural heritage solutions experienced through VR headsets, but it lacks a systematic design methodology formation and evaluation of further motivational and immersion factors that could be considered.

Cai et al. [8] have reported their assumptions on the comparison of the use of 360° video recordings and Virtual Environments in triggering a memory of the past. Their study involves the creation of a virtual house with realistic photogrammetric reconstructed objects of the dwellings used daily by local Ningbonese people since the last century. That virtual reconstruction has been compared with a high-resolution 360° video showing an old couple cooking food inside a well-preserved house located in Ningbo. Participants used an HTC Vive to interact with the virtual reconstruction that allowed interaction with virtual objects like grasping or opening drawers. The environment constructed with 360° videos had been experienced passively though a Samsung Gear device allowing only the view rotation based on head movements. The applications have been tested by 21 local participants familiar with the history of the city. The results revealed that the video felt more real, exposing the high potential of the medium, though the virtual reconstruction contributed better to familiarity and memory recollection. Therefore, there is preliminary evidence that 360° immersive video user experience issues are related to the lack of interaction in such environments. This paper attempts to address this issue by providing an analysis of design methods for 360° immersive video applications.

Another interesting study on the comparison of a VR application to 360° media is that of Boukhris et al. [5]. A user study has been conducted to compare a Virtual Reality Cultural Heritage visit to a 3D model of a Palaeolithic cave, the “Grotte de Commarque” located in the south of France, with a series of 360° pictures experienced through a VR headset. Viewers could observe the cave and explore it through teleportation mechanisms. The study findings reveal a better sense of presence offered by the 3D VR version. However, the form of 360° media used in this study is limited to 360° pictures, not video while offering minimum interaction.

None of the studies in 360° immersive video applications has experimented with navigation design approaches beyond teleportation, such as moderate motion design with locomotion (discussed in detail in Section 3). The design and effect of human actors/avatars in motivating user’s actions and increasing engagement have been studied in the context of immersive VR by Sylaiou et al. [27] and Carrozzino et al. [9] providing significant insights. It is, therefore, considered important to examine the role of human actors in user motivation for the case of 360° immersive video applications. The study presented in this paper analyses five key design aspects that should be considered at the design of interactive immersive video experiences: the narrative, the virtual scenes to be captured, the actors’ role, the navigation and the gamified design. This is followed by the evaluation of the design techniques using a testbed application for a virtual cultural heritage tour. The evaluation output led to the formation of a set of design guidelines for the creation of engaging and interactive 360° immersive video applications.

3 Three hundred sixty–degree immersive video applications design aspects

The design of 360° immersive video experiences includes two layers:

  • The experience design layer that covers all the elements that support the experience design, such as the design and creation of the required media resources, the flow of the story, the scenes’ elements and their connectivity;

  • The interaction design layer, a significant part of the design process, complementary to the experience design layer that deals with tasks related to navigation in the virtual world, interaction and system feedback, as well as the integration of gamified aspects to increase engagement.

Figure 1 presents the different requirements, challenges and guidelines that need to be addressed by those two design layers of 360° immersive video. A categorisation of design aspects and corresponding techniques is presented as a result of the methodological analysis of the design requirements at each layer.

Fig. 1
figure 1

Three hundred sixty–degree immersive video applications design aspects

Full size image

Narrative design task includes the specification of how the story unfolds, the overall story design and techniques that should be followed to increase engagement and immersion in 360° video storytelling. Aiming to increase the sense of control and give the sense of ownership to users, the technique adapted in narrative design is branching storytelling. In branching storytelling, different paths of scenes that could be visited are defined allowing users to choose their preferred path to experience a story/narrative. To achieve branching storytelling, the story design should be adapted to follow a non-linear design specification and should be used during the video production and editing phase.

Virtual scene design refers to the definition of elements that should be captured or integrated in the scenes to support the storytelling experience. With the aim to design for consistency and cognition, the scenes should depict areas and points of interest (POIs) that are contextually related in a storytelling experience. Those POIs could be for example historical buildings in a virtual city tour experience. Graphic elements such a text panel should also be integrated in each scene to provide further information about the POIs. The POIs and graphics positioning specification per scene allow the strategic design of each virtual scene and the definition of the user’s viewshed and available interaction tasks when entering each scene.

Actors’ role design involves the scripting of the actions of humans captured in the video scenes. Human actors efficiently integrated in the scene, taking the role of narrator or used to motivate the user look around, could provide an element of social engagement supporting users to immerse in the story. The human’s actions and narrative specification should be produced and used during the video production and editing process.

Navigation design is an important interaction design task for assisting viewpoint reference and scene-to-scene transition. Moderate motion is a technique that refers to user navigation in a multi-scenery environment and progressing in the story. To achieve a more realistic interaction with the 360° immersive environment and the effect of “feeling like being actually there in the virtual world” [25], walking video resources could be produced and integrated in the experience. Three hundred sixty–degree videos with locomotion should be carefully produced if intended to be experienced though HMDs. Fast camera motion in 360° videos has been reported by Tran et al. [29] to produce strong sickness effects when used in VR communication. Three hundred sixty–degree videos that are captured through steady and slow walking in an area could be used for VR world navigation, simulating a real walking experience to transition from one scene to another. Another moderate motion technique to navigate the virtual world is the direct teleportation that could give a sense of controlling better the scene-to-scene transition and empower the user. Location reference techniques should also be considered providing the user with feedback of where the POIs are placed in the scene according to her current view.

Gamified design is an efficient technique for the creation of engaging experiences. In cultural applications, educating users on historical, cultural or even environmental facts is a typical requirement. Such tasks to be engaging and effective should challenge users’ thinking, motivate them to dive deeper in the story, explore its affordances and assess their capabilities. Efficient design techniques for creating such experiences come from the area of game design. One such technique is allowing exploration and discovery by gradually revealing information to the users about observed POIs while navigating different scenes and by assessing their knowledge. Another technique has to do with increasing the sense of ownership, which can be achieved when the story unfolds in a way that allows users to achieve specific goals. Empowerment can also be achieved by allowing users to level up providing a sense of control and increasing their motivation. The design of an experience with levels could be achieved by exploring scenes and POIs gradually and providing information and feedback as the story progresses.

The defined categorisation provides a rigorous methodological approach assisting the specification of the techniques, tasks and considerations that should be followed for each layer of design of immersive video applications. The technical specification diagram provided in Fig. 2 shows the workflow and production steps of 360° immersive interactive video solutions. The techniques specification for the experience as also the interaction design layer of design results to the production of:

  • The required video scripts defining what should be captured by the videographer and what the actors should do;

  • A list of the necessary graphics that need to be produced, such UI panels imposing interaction or providing additional information to support the story flow;

  • The game logic and interaction functionality providing the required input by the user and the expected system output and updates [3].

Fig. 2
figure 2

Three hundred sixty–degree immersive video application development flow diagram

Full size image

The video content production and editing phases for the 360° video resources capturing are determined by pre-defined video scripts. The video capturing is done using a compatible 360° video camera. Following the video content, production stage is the creation of the VR scenes using a game engine such as Unity that supports application development for VR headsets such as Oculus and Samsung Gear VR. The scenes in general are created by mapping the 360° video resources to a spherical 3D object, setting up the VR cameras position according to the application scenario (what the user sees at the beginning of each scene) and importing and positioning the graphic elements (2D UI panels etc.) following the video background.

As a final process, the application functionality programming takes place based on the game and interaction logic definition leading to the VR application build.

4 Three hundred sixty–degree immersive video prototype application design for Cultural Heritage

4.1 Historic Rethymno virtual tour creation

To address the needs of the study an immersive storytelling experience, an interactive 360° video-based virtual city tour of the historical centre of Rethymno, in Greece, has been developed.Footnote 1 The application has been developed to be experienced using Oculus Rift VR headset.

The Rethymno city has been chosen for this prototype application due to its great historical importance, preserving artefacts and monuments in good condition capturing historical periods spanning from the Ottoman to the Venetian periods [13]. Those artefacts offered the opportunity for creating an interesting narrative experience by capturing city scenes with historically related POIs, as discussed earlier and outlined in Fig. 1.

Following the design of the historical tour, six short-time 360° video scenes, per area of interest, have been produced through videos captured in first person view depicting the user standing in the middle of each scene or in front of a specific POI (a historical building or artefact) to allow the exploration of the place. The recording of the 360° video content was done with a Ricoh Theta S camera device that consists of two 180° FoV cameras and is accompanied by a software editing application that allows automatic stitching, manual editing and export to .mp4 format. Those exploratory video resources have been used to create a mixed media virtual scene enriched with UI graphic elements depicting important historical textual information about the captured artefacts [13]. When the users manage to reveal this information, they can proceed to the next level of the experience. Branching storytelling is integrated in a scene following the introductory scene that offers the users the option to navigate to a series of scenes and discover the included POIs in an arbitrary sequence. The POIs are preserved historical fountains from the Ottoman period. After all POI and information have been discovered, the user is directed to the last scene where only one, the most important historically artefact, can be visited.

Two actors have been captured in the scenes: one providing introductory information and narration at the start scene, and another one for motivating and directing the user to look at specific areas in the scene to discover the artefacts to be found and reveal relevant historical information. The technique of using an actor to motivate the user to look around belongs to the provision of location reference of the POIs in the 360° scenes as a solution to navigation design task. Another similar technique to support location reference is through the use of graphic UI elements. Graphic vectors have therefore been integrated in a specific scene to allow the comparison of techniques.

Different moderate motion techniques to support the transition from one scene to another were incorporated in the design process, such as walking simulation and direct teleportation. The walking simulation refers to the capturing and integration of videos where the subject has the sense of being moved through walking down an alley before moving to the next scene. In contrast, with direct teleportation, the user is instantly transferred to another area through direct change of static captured video resources.

The experience had been designed as a gamified, educational tour, revealing information about the city in a fun and engaging way. The player is informed by a narrator that will play the role of an Ottoman soldier missioned to collect as much water possible from several historical fountains of the Ottoman period, which remain in the city, and carry it to the most important fountain of the city, the Rimondi fountain [13]. The role of the Ottoman soldier has been mentioned with the aim to create empathy with the mission and the context of the story as there is no evidence on user’s embodiment when experiencing the video scenes captured in first person view. In this journey, the user has to spot three fountains, discover relevant historical information and reply to a set of questions in order to collect water for the Rimondi fountain (the final spot). The game ends after the user has visited all fountains and reached the Rimondi area, where the players are addressed with a gold, a silver or a bronze badge of the Ottoman citizen of Rethymno according to the points collected throughout the tour.

Feedback on user progress is given through a water collector indicator UI element showing the users’ current score. In that way, the users are learning about the history of the city by completing a set of tasks and challenges that allow the interaction with the VR environment. Challenges are continuously presented to the players at each level of the game, keeping them curious while testing and applying their knowledge. Addressing challenges makes people feel they have earned their achievement giving them the sense of accomplishment which is one of the eight core drives of gamification according to the Octalysis gamification framework [10].

The logic of the overall gamified experience is designed based on the concept of exploratory games that allows the users to freely navigate and visit several stages of the game by experiencing different narratives until they identify and complete all challenges presented [12]. The exploratory approach triggers the users’ curiosity motivating them to master the rules and affordances of the game by supporting them to level up and advance in the game, making the whole experience more engaging.

4.2 Testbed design approach for hypothesis evaluation

This case study investigates the effect of different design techniques integrated in the 360° immersive video historic Rethymno virtual tour with respect to immersion and engagement levels achieved. The main objectives of this study were as follows:

  • The evaluation of the overall design approach followed in terms of engagement and immersion offered;

  • The comparative assessment of the effectiveness of the techniques applied for immersive moderate motion design (walking simulation—video resources with locomotion vs direct teleportation);

  • The comparative assessment of the techniques applied for navigation with location reference design (user of human actor vs graphic UI elements);

  • The evaluation of the educational potentials and engagement level offered through the gamification of a Cultural Heritage storytelling experience using the 360° video medium.

In order to serve the purpose of this study, the testbed (the 360° immersive video historic Rethymno virtual tour) has been designed following the experience and interaction design layer technique categorisation (outlined in Section 3). Those are depicted in different colour codes in Table 1 (green for navigation-location reference design, orange for moderate motion design, purple for gamified design and blue for the motivational elements that may trigger user choice). Each scene of the testbed integrates relevant design elements falling in different categories of the experience and interaction design layers (outlined in Section 3) captured in (Fig. 1). The top row of Table 1 indicates the scenes and the scene content, while the columns correspond to the design elements integrated per scene.

Table 1 Design techniques applied at each scene of the 360° immersive video historic Rethymno virtual tour
Full size table

For the development of the overall game logic and in order to track the users’ actions and update the game scenes and UI elements accordingly, while providing accurate feedback, the following variables have been created:

  • Current score—recording the users’ achieved score as progressing in the game;

  • User answer selection—defining if the correct answer was spotted immediately, after one wrong selection or after two wrong selections;

  • Final score—defining the final achievement level and badge assignment;

  • Current scene—indicating the scene the user is viewing at a time;

  • Current path selection—defining the next path choices that should be revealed;

  • Current path options—defining the available path options;

  • Time spent until spotting fountain with human-assisted navigation;

  • Time spent until spotting fountain with UI-assisted navigation;

  • Time spent until spotting fountain with no navigation support.

4.3 Immersive video interactive scene design

The 360° immersive video historic Rethymno virtual tour application was designed to serve a set of test scenarios that allow the evaluation and comparative study of different design techniques. Those scenarios were running at each of the six different scenes of the interactive storytelling tour experience.

  1. Scene 1:

    Intro to mission by a human narrator

At the first stage, the user is placed at a historical courtyard where a Turkish woman, serving the role of the narrator, is welcoming them, talking about the history of the city and explaining their mission (see Fig. 3a.

  1. Scene 2:

    Path selection

Fig. 3
figure 3

Scenes and design elements consisting the Rethymno 360° immersive video virtual tour

Full size image

Moving on, the user is transferred to a new virtual scene, at the old city of Rethymno, in front of a crossroad path, where introductory UI panels appear presenting the first task asking the user to select a fountain to visit (see Fig. 3b). The interaction with the UI elements is gaze-based. This means that the UI buttons appearing in the scene are triggered by focusing on them for a few seconds followed by a green filling effect as depicted in the figure. A short walking video follows resembling locomotion and moving the user towards the path selected transferring them to the scene of the corresponding fountain.

There are three fountains that the users should visit at a sequence of their preference according to their selections. Those fountains are placed in different areas of the historical city centre dated from 1863 [11]:

  • Fountain 1: the fountain erected in Patriarchou Grigoriou Street by Kasim Bey;

  • Fountain 2: the fountain in Prevelaki Street erected by Yunus Aga’s son, Ethem Bey;

  • Fountain 3: the fountain at the corner of Smyrnis Street and Koronaiou Street built by Osman Efendi.

At each of the fountain scenes, the users experience a different technique that intends to guide them to spot the historical fountain by motivating them to rotate their view.

  1. Scene 3:

    Fountain 1—human-assisted navigation

At scene 3, the navigation is supported by a human actor, meaning that there is a person in the scene creating a human contact with the subject who turns towards the fountain to motivate the user to also turn and figure out what she is looking at (see Fig. 3c). When the user turns towards that direction, a panel with a challenge in the form of multiple answers appears, providing also further instructions on how to move in the story.

The user has to select a UI button that appears below the question to choose the correct answer. When an answer is wrong, the button is marked red to indicate error, and the user has to select another button until the correct one is revealed and turned green, as shown in (Fig. 3e).

A congratulations text communication panel follows revealing the number of litres of water gained. The water litres are calculated based on the number of attempts the users take to answer a question correctly (30 points are gained if a question is answered correctly at first attempt, 20 points at second attempt and 10 points when it is revealed by the system) (Fig. 3f). The user should then select to collect the water gained, and a graphic animated pot appears to be filled gradually based on the litres translated percentage (Fig. 3(h)).

As a follow up step, the panels illustrate the available options for choosing the next fountain to be visited (Fig. 3g). The user has to select the teleportation button (that has a relative graphic design different from the walking simulation) to be transferred to the selected scene, meaning that the corresponding video resource is enabled and played.

  1. Scene 4:

    Fountain 2—UI-assisted navigation

At scene 4, the user experiences a new navigation assistance technique in the form of dynamically triggered pointing vectors that fade when the user turns to the fountains position (as shown in Fig. 3d). A new multiple-choice question appears then and when the correct answer is revealed, the scene is updated showing the score indicator element. The next panel directs the user to be teleported to the final fountain area depending on the previous path followed.

  1. Scene 5:

    Fountain 3—no navigation support

At scene 5, no navigation technique support is provided, and the user should spot the fountain on its own. Moreover, this scene is enhanced by digital background music and not the natural and realistic sounds captured in the video resource, as already experienced in the previous scenes.

  1. Scene 6:

    Rimondi—final scene

When all the fountains have been visited and all questions are answered, the user is teleported to the last scene of the experience. This is the most iconic fountain of the city of Rethymno, the Rimondi fountain, which is shown functional with water running from its three tabs, indicating the mission accomplishment. A text panel appears informing the user that the mission has been completed and that he/she has helped in the hydration problem of the city through a certain amount of water according to the score level achieved (as shown in Fig. 3h). Based on the final score, a citizen badge is assigned to the user that is either bronze (up to 30 l), silver (40–60 l) or gold (70 and over litres).

5 Experimental methods

The study was designed to evaluate the effectiveness and user acceptance of the design techniques for 360° immersive video applications (see Section 3) that were integrated in the Rethymno virtual tour. The successful application of the design is marked by the:

  • Completing the assigned tasks (effectiveness), little effort (efficiency) and satisfaction [19];

  • Cognitive processing (accessing, interpreting and responding to) the information conveyed by the virtual environment [23].

In addition, the study was planned to allow the assessment of the levels of engagement and immersion achieved through the design approach followed. The success of the design techniques could lead to the development of a set of guidelines for the design of engaging and interactive 360° immersive video experiences.

The experimental evaluation activities using the 360° immersive video historic Rethymno virtual tour application included lab-based testing.

The prototype design that has been described in Section 4 also supports conducting a comparative study of user performance when different design elements are applied.

5.1 Experimental conditions

The experiments run in the mixed reality lab at the School of Computer Science & Engineering, at the University of Westminster that provided the required VR equipment and a safe environment for conducting the study (Fig. 4).

Fig. 4
figure 4

Participant testing the application using Oculus

Full size image

The VR equipment used were as follows:

  • One PC able to handle the tethered Oculus VR headset (1080 × 1200 per eye resolution, 110° FoV, 90 Hz refresh rate);

  • An audio recording device to record interviews followed each session.

The study duration was 3 days, dedicating 30-min timeslots for each participant. All users have tried the application using the Oculus VR headset. The ordering of experienced scenes was random as it relied on the user’s choices during the branching narrative. Each VR tour experience lasts approximately 7 min. Prior to the main session, the researcher introduced the scope of the study and handed out an information sheet and a consent form to be completed. At the end of each session, the participants were asked to complete a scaled 1–5 questionnaire that took approximately 5–10 min to be completed that served the purpose of collecting data related to the participants’ experience in terms of immersion and engagement. The session ended with a short interview of the participants about their overall experience that lasted approximately 5–10 min. The study complies with the University of Westminster ethics experimental protocol.

5.2 Data collection methods

The data collection process included a set of different and comprehensive methods.

A logging mechanism integrated in the Rethymno 360° immersive video application was recording quantitative data during runtime to study each participant’s response time in completing the task of identifying the fountains. This has been accomplished through a script, integrated in the Oculus application that calculated the time from entering a scene (and the reveal of the navigation assistance mechanism—human eye contact motivation/graphic vectors/baseline) up to point of facing the fountain contained in the scene and logged to a .csv file.

Structured questionnaires with two sections recorded as follows:

  1. a)

    Demographic data and subject preferences;

  2. b)

    Data capturing the overall user experience.

The first part of the questionnaire was focusing on collecting data on the users’ prior experience, habits, age range and background.

The second part of the questionnaire consisted of 20 questions targeting the collection of data related to:

  • Sense of presence and user satisfaction of the immersive experience;

  • Preference of design techniques and elements integrated in the different scenes;

  • User acceptance of techniques applied;

  • User level of engagement with the overall experience.

Each question has been defined corresponding to a specific factor measurement related to:

  • Immersion, such as sense of presence, realism, naturalness, consistency and loss of time awareness; and

  • Engagement, such as endurability, confidence, novelty and focus of attention.

A 5-Likert scale (1: strongly agree, 5: strongly disagree) that uses fixed choice response formats designed to measure attitudes or opinions [6, 7] is used.

Note-taking during conducting the test collected qualitative information about the participants overall experience, marking down issues of malfunction and difficulties in using the application and the required equipment.

Face-to-face interviews following each task capture the participants’ comments about their overall experience.

5.3 Participants

Thirty-eight (38) users (23 males, 15 females) with ages spanning from 18 to 50+ participated in the study on a voluntary basis invited through e-mails sent to University lists. Thirty-nine and one-half percent of the participants were under 30 (18–30 years) while the 60.5% of them were between 31 up to 50+. Most of the participants (71.1%) were undergraduate, post-graduate students, academic staff and professionals with a background in computer science and related studies (design, HCI) while the rest had no technology background, such as psychologists or administrative staff.

More than half of the subjects (57.9%) were frequent game players, playing games at least on a monthly basis and 71.1% has had at least one VR experience in the past.

6 Evaluation results

The questionnaire generated non-parametric and categorical quantitative data that were not only ordinal but also nominal in some cases (gender etc.). In contrast, the data collected from the system logging mechanisms generated scale measurements and were analysed with most common non-parametric analysis tests equivalent to repeated measure ANOVA tests. The sections below provide a more thorough analysis of the quantitative data, followed by an interpretation of the outcomes, concluding with a reporting section on the qualitative data collected during the after-tests interviews.

6.1 Immersion evaluation

To evaluate the level of immersion achieved using the Rethymno 360° immersive video virtual tour (see Section 3), Fig. 5 shows the Likert scale means with 95% confidence intervals of questions related to sense of presence, realism, naturalness, consistency and loss of time awareness. In terms of realism of the experience (mean value 2.26 close to 2-agree), “feeling like actually being there” (presence) and the achieved level of disconnection from the real world, the results confirm the hypothesis that the design that follows the proposed methodological considerations (see Section 3) provides a satisfying level of immersive experience. The results related to loosing time awareness are neutral; the participants’ replies vary, so no conclusions can be drawn related to the effect of the prototype design to offer such a sense. However, it is worth stating that besides the effect of locomotion integrated in the form of simulating movement through waking in the scene, the results indicated that participants did not feel dizzy. Dizziness is a feeling that breaks immersion [18]. This was achieved by capturing 360° video resources by holding the camera and walking slowly and steadily. Therefore, its potential for providing comfortable moderate motion while exploring 360° video is high.

Fig. 5
figure 5

Immersion questions with 95% confidence intervals

Full size image

6.2 Engagement evaluation

In terms of engagement, the design of the Rethymno 360° immersive video virtual tour addresses the users’ expectations. This is indicated by all related data from questions measuring factors associated to enjoyment, control, excitement, endurability, confidence, novelty and focus of attention had positive results (mean values close to 2—agree with error bars overlapping) as depicted in Fig. 6. Overall, the design approach showed good potentials in providing engaging experiences through the integration of branching narratives, gamified techniques and moderate motion effects.

Fig. 6
figure 6

Engagement questions with 95% confidence intervals

Full size image

6.3 Moderate motion techniques comparative results

As depicted in Fig. 7, the results on questions related to the user’s acceptability of the integrated navigation techniques from one scene to another indicate a preference towards the teleportation mechanism. The mean of the users’ preference for walking simulation is 3.46—closer to neutral. Thus, no valid assumptions can be formed regarding the most preferred technique for scene transition, but it can be concluded that the teleportation design offers a smooth and comfortable solution. Provided the results are neutral and not negative on the walking simulation (mean value 3.22 close to 3—neutral response), we can assume that there is potential for an acceptance also of this technique especially if the production of the videos is done with professional stabilization and higher quality equipment.

Fig. 7
figure 7

Moderate motion with 95% confidence intervals

Full size image

6.4 Navigation techniques comparative results

Two navigation assistance techniques have been integrated in the Rethymno 360° immersive video virtual tour testbed:

  • Human motivation: the integration of an actor in the video resource motivating the users to turn and look towards a direction the actor is looking by first creating human contact with the users and then turning and looking towards a POI;

  • Vectors UI: the placement of directional graphic vectors that point to the direction of a POI and fade when the user turns towards the direction they point.

The results regarding the effectiveness of both techniques are very positive, with most of the participants stating that they strongly agree with the statement that they were motivated to change their view by those techniques (as shown in Fig. 8). In terms of user preference between those two techniques, there is no indication that users liked most one technique to another as the mean value of the corresponding question results is 3.1 showing neutrality. When participants have been asked if they would prefer to have no assistance and explore the environment at their own pace, the results were close to neutral (mean = 3.44), and therefore, no accurate assumption can be formed based exclusively on that data.

Fig. 8
figure 8

Navigation means with 95% confidence intervals

Full size image

To further evaluate the efficiency of the navigation assistance techniques, a system log mechanism has been integrated in the testbed for Oculus, in scenes 3 (human motivation) and 4 (vectors UI), while in scene 5, no navigation assistance technique is integrated, forming the baseline. The log mechanism records the response time that takes the user to turn towards the POI (fountain) pointed by the incorporated navigation assistance technique. The data indicated that in scenes with navigation mechanisms, the participants’ response time was significantly shorter compared with a scene with no navigation mechanism (referred as baseline/no support in Table 5).

By comparing the different navigation mechanisms’ response time means in Table 2 below, we see that the most efficient technique seems to be the human-guided navigation support, with a mean value of 3.53 s. The less effective navigation mechanism is the baseline solution where no navigation assistance technique has been integrated, with a mean value of 10 s approximately.

Table 2 Mean, standard deviation and variance of the log data for the different navigation techniques
Full size table

However, the log time raw data is not normally distributed based on the significance results of Kolmogorov-Smirnov and Shapiro-Wilk tests (p = 0) (see Table 3). Thus, a full factorial ANOVA analysis is not allowed, and a non-parametric test should be performed. An equivalent non-parametric test is the Friedman test that does not include though interaction analysis.

Table 3 Normality test of user time response to turn towards a POI driven by different navigation techniques
Full size table

The non-parametric Friedman test exploring the differences among repeated measures on time revealed a significant (p = 0) Chi-square value of 22.158 (see Table 4). This points out for a significant difference between the two navigation mechanisms.

Table 4 Non-parametric Friedman test analysis for time repeated measures on the three navigation design techniques
Full size table

A post hoc analysis has been performed for pairwise comparisons using Wilcoxon signed-rank test among the different navigation assistance techniques (see Table 5). From the Wilcoxon signed-rank pairwise comparison analysis, it is evident that there is significant difference between the time performed with the vectors UI technique in comparison with the baseline/no support (p = 0.003 < 0.05). The users also performed significantly faster with the human motivation technique than with baseline/no support (p = 0 < 0.05). There is no significant difference through on time to perform the task between the two technique human motivation—vectors UI (p = 0.112 > 0.05).

Table 5 Wilcoxon-signed rank pairwise analysis of time
Full size table

We can assume based on the repeated measure comparison results that the navigation mechanisms were both efficient comparing them to baseline/no, but no conclusion can be made on which technique (human vs vectors) was most efficient in terms of navigation.

6.5 Narrative design results

The user acceptance level of the branching storytelling technique that has been adapted in the design of the Rethymno 360° immersive video virtual tour testbed, empowering the users with the choice to follow their preferred path to visit the next fountain (see Scene 2 in Section 4.3 and Fig. 3) was high. Most of the participants strongly agreed or agreed (30/38) with the statement of liking the freedom to choose their own path with none of them disagreeing.

6.6 Qualitative data analysis

Qualitative data have been collected while conducting the study through direct observation and notetaking and at a follow up interview after the completion of the session (see Section 4.2). All the participants commented that the virtual experience was interesting, and they would like to have similar experiences in the future as also that they were engaged and remained focused throughout the experience. They also pointed out that it was in general, a comfortable experience with only a few of participants commenting that they felt a bit dizzy at some point when experiencing locomotion. In general, the participants commented that they would prefer to have a better quality of video, which was expected as the camera resolution used for the creation of the video resources was low. Some participants commented also on the position of the UI text panels by stating that the multiple-choice question elements were cutting out the view from the fountains.

A significant number of the participants suggested that they would prefer a more relaxed experience of increased realism by using more human actors in different stages of the game. In addition, they suggested creating short walking simulations with intermediate tasks to feel like they were exploring the area at their own pace and feel more like truly walking in the streets of the historical city centre. Some of the participants also pointed out that they would like to be able to find clues that could help them in finding the right answer to the questions addressed and also that this could lead to an effective educational solution that they would enjoy.

7 Design guidelines

The interpretation of the outcomes of this study allowed the formation of a set of preliminary assumptions on the effectiveness towards engagement and immersion of the design techniques integrated at 360° immersive Cultural Heritage video applications. Those assumptions could be generalizable to other forms of immersive video virtual tour applications. In order to effectively integrate those techniques, a set of design guidelines (DGs) outlined below is proposed. Those DGs are presented in three parts (see Table 6): requirement/observation; design guideline; application example, following a model suggested that DGs need to be precise and providing examples of use in order to reduce the chances of the guideline being too vague, or conflicting [14, 22]. The DGs are grouped per immersive video design aspect.

Table 6 Design guidelines for 360° immersive video applications
Full size table

8 Conclusion

The study presented in this paper attempted to introduce a set of design tasks and techniques that should be considered at the experience and interaction design layers of the design process of creating 360° immersive video applications of Cultural Heritage tours. Those design aspects were considered in the creation of a testbed application of a virtual tour in the historical city of Rethymno, Greece. The Cultural Heritage virtual testbed application has been subjected to lab-studies involving real users allowing the evaluation of the proposed techniques. The results of the study led to the definition of a set of design guidelines suggested for the creation of immersive video interactive applications.

The evaluation results indicated that the methodology followed for the design of the 360° immersive video Rethymno virtual tour led to high level of engagement perceived by the participants. The immersion level was also satisfying with participants feeling comfortable throughout the experience that combined two moderate motion techniques, including 360° video captured with locomotion and teleportation, showing preference to teleportation. The integrated mechanisms for supporting navigation in the 360° virtual scenes and identifying POIs were significantly more efficient comparing with the absence of navigation support. The first technique was based on the use of a human actor creating eye contact with the subjects/participants to motivate them to look at the same direction with them, while the second was based on the use of graphic UI vectors pointing towards a POI.

A gamified design was also followed with the aim to engage the users further with the historical virtual tour experience and provide enhanced interaction with the video-based virtual scene. The tour introduced the mission of visiting the preserved fountains of Rethymno to collect water, experiencing in a way the citizen daily activities of that period of time. In each scene of the tour, a fountain was visited, and participants were presented with a challenge in the form of multiple answer questions that assigned participants with points in the form of litres of collected water. The participants were learning about the history of the city in a fun and entertaining way even in the cases they answered a question incorrectly. The proposed design methodology for 360° immersive video experiences offers a new approach for delivering informative and entertaining Cultural Heritage virtual tours. The results of this study revealed the high potentials of the immersive video medium in the area of Virtual Cultural Heritage digital application design.

Moreover, future studies are suggested to explore other design methods and techniques that could be followed in 360° immersive video UI design such as the use of cognitive UI map visualizations for 360° in-scene navigation. Different UI design approaches for VR should also be investigated on their pertinence in the case of 360° immersive video. The proposed techniques should also be evaluated in future studies assessing usability by isolating the assessment of each design mechanism and performing further comparative studies.