Creating an Interactive Urban Traffic System for the Simulation of Different Traffic Scenarios

Abstract

The social and political efforts to fight climate change have contributed to a re-thinking of traffic systems, especially in urban areas under constant transformation. To simulate and visualize planning scenarios of urban traffic systems in a realistic way, the possibilities of virtual 3D environments have regularly been used. The modern potentials of (immersive) virtual reality, however, still require exploration, evaluation, and further development. Using the game engine Unity, an immersive virtual environment was developed to visualize and experience dynamic traffic conditions of a highly dense urban area. The case study is based on the characteristic model of a Central European city (not a representation of a real city), which brings together the specific considerations of urban traffic, such as mirroring the complex interplay of pedestrians as well as individual and public transport. This contribution has an applied methodological focus and considers possibilities as well as difficulties in the design of a reliably running (open-end) traffic system. The applied tool for the creation of a modular and customizable traffic system in Unity resulted in a traffic system that is capable of reacting to the individual behavior of the user (including the individualized motion of the avatar), without leading to accidents or uncorrectable traffic jams. Therefore, the tool used could be a valuable option for any developer of immersive virtual environments in Unity to equip these immersive virtual environments with a traffic system, without the use of additional third-party software.

1. Introduction

The ongoing worldwide debates on the consequences of climate change, digitization, and urban migration have led to new political and planning concepts for governing and redesigning cities and their urban systems. An important field of action is inner-city traffic, whose emissions (pollutant emissions and noise) and dynamics have a major impact on the lives of citizens [1,2,3]. New approaches to transport systems in the future that pursue smart and climate-neutral goals are leading to significant changes in the urban landscape; for example, the replacement of combustion engines with electric motors, the increase in public transport with a decrease in cars at the same time, the increase in bicycle traffic with more bicycle paths, new approaches in unmanned parcel delivery, more charging stations for electric mobility, more greenery at the roadside, etc.

In order to simulate and plan this complex interaction of new approaches in the urban traffic system, modern approaches to visualization in virtual reality (VR) are suitable. VR-based visualizations enable different actors in the planning process to experience the entire construct in an immersive approach. In addition, by assigning rules of motion and behavior to each relevant traffic object, a system can be created (and changed in terms of individual parameters) that simulates the multisensory impact of inner-city traffic.

This article aims to present an example of a transferable VR-based traffic system in the model (typical traffic situation at an intersection) of a Central European city. After a summary of VR-based approaches to urban traffic simulation (Section 2), the core steps and characteristics of a new approach (Section 3) are presented, and the limitations of the approach (Section 4) are discussed. In terms of methodology, the approach presented in this article is based on the current possibilities of the game engine Unity.

2. Virtual Reality in Urban Traffic Simulations

The idea of simulating urban traffic systems, in the sense of “traffic as a simulation of object” [4], has a long tradition and has made regular advances with new technology and methods (see also [5]). Several research efforts have been made to apply technological and methodological innovations to simulate traffic situations, such as CAD and GIS [6,7,8], parallel computing [9], and 3D modelling [10,11]. With increasing computing power, the number of objects has also increased within virtual traffic models cf. [12,13,14]. Approaches to the use of virtual reality began to play an increasingly important role in the 2010s (e.g., [15]).

Today, VR technologies are known in geoinformation sciences for the development of immersive environments and have so far only been used sporadically for traffic simulation. To bring together traffic planning and the design of sustainable urban spaces, bike simulators that facilitate immersive VR have recently been developed [16,17,18]. With respect to inner-city automobile traffic, few studies are available to date that address specific components of VR-based simulations.

Edler et al. [19] discussed approaches to the dynamic visualization of 3D sound for VR-based road traffic simulations. Beyond the visualization of urban soundscape ecology, Wu et al. [20] used 3D sound in immersive VR to study pedestrian decisions in intersection situations. Using deep learning approaches, Kalatian and Farooq [21] attempted to analyze pedestrian and vehicle interactions in immersive VR. Currently, research on the technical development of interacting transportation systems in immersive VR is scarce. This study follows up on this by presenting an approach to an interacting traffic system in immersive VR.

When it comes to traffic simulations within the game engine Unity, which became free to use in 2015 for personal users, there is a lack of publications dealing with the creation of transport systems from a non-expert developer perspective. Rundel and De Amicis [22] created a workflow based on a combination of different proprietary and freely available stand-alone software to implement a traffic system into the game engine Unity. The use of proprietary software in this workflow might be a hurdle for developers who want to benefit from freely available software such as Unity and also implement a traffic system without the use of additional third-party software. Liao et al. [23] used the freely available traffic simulation software SUMO to implement traffic into Unity, but focused on the analysis than the implementation and visualization of the traffic system. Chen and De Luca [24] used a visual programming language to simulate random traffic and autonomous driving.

Thus, in this paper, the proposed method provides a cost-effective solution to allow developers of immersive virtual environments without a technical background in traffic simulation to quickly add traffic to their immersive virtual environments in Unity, without the need to learn and use additional third-party (proprietary) software.

3. New Approach to Urban Traffic Simulation in (Immersive) VR

For the development and creation of a 3D environment in the form of an exemplary inner-city intersection situation, the game engine Unity was used in this study. The future purpose of this 3D environment is to use it in geography classes to familiarize students with possible sustainable urban development concepts. The creation of additional 3D objects was realized using the Blender software. Unity provides developers with the ability to contribute their own 3D models and compatible software to the game engine using the Unity Asset Store. The development and implementation of an urban traffic system were performed using the “Simple Traffic System” asset, available in the Unity Asset Store by the developer TurnTheGameOn. This asset offers the possibility to build a modular traffic system with the help of a dedicated editor.

Locomotion in the virtual environment took place through a keyboard and mouse controllable first-person avatar. The application was released as Web Graphics Library (WebGL), which creates the possibility to run and use the application in a web browser. The 3D environment was additionally developed to be experienced with a head-mounted display and controller using teleportation as the type of locomotion. The implementation was performed using SteamVR, and the virtual reality application was released as a Windows standalone.

The initial point for the development of an urban traffic situation inside an immersive virtual environment was the idea of an intersection situation that can be found within the traffic network of a Central European city. The representation of the city in the virtual environment itself is not built on any real existing urban environment. It is considered a representative model. The intersection situation consists of a main road with four motorized traffic lanes and one lane for a tram. In addition, there are two side streets for motorized traffic (Figure 1). Around the streets, the environment is modelled with buildings (perimeter block development). The focus of transportation planning for the road network is on motorized traffic.

3.1. Requirements for Object Interaction in the Traffic System

The preconditions of the possible interactions to be applied in the transport system should be defined in advance of development. The implementation of the simulated virtual motorized traffic was performed after the virtual environment was created. The traffic system consists of vehicles, traffic lights, the traffic network (represented by interconnected waypoint routes), and an avatar that interacts with each other. The interaction between the vehicles should take place in such a way that no accidents occur. This means that vehicles should recognize other vehicles and also initiate the braking process when the vehicle in front of them brakes. In addition, the vehicles should be able to interact with the traffic light system. The traffic light system component is expected to ensure that vehicles react in accordance with the current traffic light circuit. Since the virtual environment can be experienced through a first-person avatar and, accordingly, the road can be walked on, the vehicles should interact with the avatar in such a way that the avatar is not run over by vehicles. This requires that the traffic system is able to react to individual user motion represented by the avatar. For example, the vehicles must be able to recognize an avatar when it is too close to them in order to stop and avoid accidents. After the actions take place based on the avatar’s position, the traffic has to continue without producing uncorrectable traffic problems (accidents, traffic jams at the intersection, and obstruction of the traffic routes of trams or cyclists). The traffic system in its entirety should run in a loop without interruptions (open-end). Therefore, it was defined that the traffic lights have a fixed continuous running circuit, which cannot be altered by the actions of the avatar.

The walkability of the virtual environment by the avatar is limited to the area of the intersection situation. Moving further is restricted by an invisible wall with a collider attached to it. When the avatar is navigated into the inaccessible area, the limit of the freely explorable environment becomes visible through an impenetrable red transparent wall, and an acoustic signal is then played. The walkable virtual environment is shown in Figure 2. The figure indicates that the road network goes beyond the walkable area for the avatar. The interaction features for the educational learning tasks, which are connected to interactable 3D objects, can be found in the area accessible to the avatar. Vehicles are loaded from areas that are not accessible and visible to the avatar. When the player is looking towards the areas that are not accessible, the impression is simulated that the environment is larger than it really is. For example, a train runs along the end of the main street. This is to give the impression that the virtual environment and the traffic is not limited to the intersection.

3.2. Creating a Road Network, Road Waypoints, and a Waypoint Route

As the basis of the road network, it is necessary to equip the 3D-modelled roads of the virtual environment with waypoints. These waypoints represent the traffic network on which the vehicles should drive. For the creation of waypoint routes, the asset offers a dedicated editor. First, an AITrafficController is loaded into the unity scene using the editor button. This AITrafficController controls the other components of the traffic system. Then, using the AITrafficWaypointRoute button of the user interface, a waypoint route is loaded into the unity scene. Each waypoint route consists of at least two connected waypoints. The waypoints are manually inserted into the environment. The first waypoint is the start point and the last waypoint is the endpoint of a road in the traffic system, as shown in red in Figure 3. In order to connect different waypoint routes, the asset provides a configuration mode that allows connecting the endpoints of one waypoint route to the starting point of another.

On every waypoint, the speed of the vehicles driving on the route from that waypoint to the next waypoint can be set via the script settings in Unity. To create a waypoint in the virtual environment, a so-called AITrafficController is loaded into the Unity scene via the tool window, which controls the movement of the cars along the created routes. After that, an AITrafficWaypointRoute has to be loaded via the tool window. It contains an array of waypoints that can be created by holding down the shift key and clicking the left mouse button. It is possible to create individual waypoints for waypoint routes manually.

Since the vehicles are to be loaded onto the waypoint routes outside the visibility range of the avatar, multiple connected waypoint routes were created for this purpose. Since the accessibility of the virtual environment by the avatar was limited to the intersection, the traffic system was not interconnected by several connected waypoint routes. Vehicles were loaded into the traffic system through those waypoint routes that were pointing towards the intersection. When the vehicles reached the end of the waypoint routes, they were loaded back to their starting waypoint route and drove the route using the predefined path.

3.3. Traffic Lights

As a next step, the traffic lights are integrated into the virtual environment in the waypoint network that has been created so far (Figure 4). Once a waypoint route is implemented into the unity scene, it is possible to connect traffic lights to them via the dedicated editor window. For this purpose, an AITrafficLightManager is loaded into the scene that consists of a Unity Game Object called AITrafficLight. The Game Object AITrafficLight contains three separate objects that represent the traffic light colors red, yellow, and green as a mesh. Using the edit function of the tool window, the AITrafficLight is connected to the endpoints of a waypoint route. These endpoints serve as stopping points at which the vehicles stop when the traffic light phase displays red or yellow. In the settings of the AITrafficManager, the time in seconds for the red, yellow, and green phases can be set for each created AITrafficLight. In addition, traffic light cycles can be set according to the sequence in which the respective pairs of traffic lights execute their traffic light phase. Once the traffic light circuit and the sequential order of the traffic lights have been set, the traffic lights run open-end when the application is started.

3.4. Vehicles

After the waypoints have been connected to the traffic lights, the vehicles are loaded into the waypoint routes where the vehicles are to appear and travel accordingly along the predefined route. Vehicles are the components in the traffic system that move along the predefined waypoints of the waypoint routes, waypoint by waypoint, and are reloaded to their original waypoint route when the vehicles reach the last waypoint. Thus, the vehicles consist of a modelled body and tires, which are equipped with Unity’s own wheel colliders. In order to detect obstacles (vehicles or avatars) when driving along the waypoints of the interconnected waypoint routes and to stop accordingly after detecting them, the vehicles are equipped with three colliders. Two are located on the left and right sides of the vehicle and one is in front of the vehicle.

To integrate the vehicles into the already built traffic system, the 3D models of the vehicles are connected to a waypoint route for this purpose. In order to implement them, the waypoint routes have an array of Unity Game Objects attached to them, into which the vehicle models can be loaded via drag and drop.

4. Discussion: Technical Limitations and Solutions

Theoretically, the implementation and connection of the individual components of the asset should result in a functional urban traffic system that runs open-end without errors. However, issues have occurred in the development process of the traffic system that occasionally brought it to a complete breakdown, as shown for example in Figure 5. In this particular case, traffic accidents occurred in the areas of the starting waypoint routes used to load vehicles. This error occurred when two vehicles took different routes on turning points but reached the endpoints of the different waypoint routes at the same time, which also caused them to be loaded to their starting waypoint route at the same time. The cause of this error was that the vehicles were loaded to their starting waypoint routes at the same time. This caused these vehicles to collide with each other, fall over and become unable to drive, leading to a blockage of the road, which prevented other vehicles from continuing their route, since these crashed vehicles were recognized as obstacles. This highly relevant issue counteracting the idea of an open-end loop could not be handled by the opportunities of the asset itself. It required the customized implementation of an adjustment.

To solve this issue, a collider was placed in the location of the waypoint routes used to load the vehicles. Additionally, blockades were placed at the ends of the waypoints. The collider at the loading waypoint routes is used to check whether there are currently any vehicles in the waypoint route area. If this is the case, the deployed collider at the end waypoint routes is activated and prevents vehicles from reaching the endpoint and being loaded to the starting point of their waypoint route. Other solutions to this error, such as disabling the vehicle colliders while on the starting waypoint route, are imaginable.

Another issue is that by actively moving the avatar into the interaction through the user, a gridlock of the traffic system can be triggered because the avatar is set to be recognized by the vehicles as an obstacle and ensures that the vehicles stop. If the avatar is placed within the intersection in such a way that the vehicles stop and block each other, since the avatar is detected as an obstacle, this leads to a traffic gridlock situation as shown in Figure 6. Such a traffic gridlock would not resolve itself and would persist until the application is restarted.

To resolve such gridlocks caused by irrational user behavior, vehicles that have not moved for a fixed time period are loaded back to their starting waypoint routes and disappear from their current position in the environment. The waiting time has been set longer than the stop time of the traffic light phase so that the vehicles waiting at the traffic lights are not mistakenly loaded to their starting waypoint routes. Considering that the vehicles disappear from the environment after the waiting time has expired, it could lead to a negative impact on the user’s immersion if the disappearance is perceived.

There could be other difficulties in implementing a transportation system that is not encountered in the application example presented here. The traffic system implemented here did not consist of a closed system of waypoint routes, since the area of the virtual environment was limited to the intersection. Especially in the case of a further developed, more complex, and closed traffic network with more traffic volume, further sources of error and difficulties would be imaginable. Further developments in traffic systems should deal with the irrational behavior of users in order to avoid errors triggered by irrational behavior. In addition, studies are needed to identify the key factors of accidents and errors in traffic simulations as has been carried out by Liu et al. [25], who elaborated on the key factors of the severity of traffic accidents between automobiles and two-wheelers.

Despite the errors and the described limitations of the traffic system, the implementation into the already existing virtual environment in Unity is user-friendly due to the modular structure of the individual components and the user interface. Developers who value and prefer a fast and user-friendly implementation of a traffic system with a focus on traffic visualization could benefit from this tool without relying on other additional software. Due to the availability of the source code, adaptations by developers who want to go beyond the sole purpose of traffic visualization and extend the tool for analytical application purposes are conceivable.

5. Summary

The simulation of traffic systems in virtual environments has been a topic attracting a multidisciplinary research community for decades [4,18]. With new political and planning concepts for urban transformation, as well as with the increasing computational power and software solutions, [13,14], the complexity and applications of traffic simulations have increased.

The current opportunities for developing realistic, immersive, and intelligent VR-based traffic systems in Game Engines provide a new dimension of interaction. Individual motion paths of the user’s ‘virtual ego’ (avatar) require a permanent interaction and reaction of other animated objects within the system. This leads to an individual chain of follow-up reactions. To avoid a collapse of the permanent loop of the traffic system, rules must be defined in the development phase that guarantee a running system and do not restrict the freedom of user motion and the perception of a realistic scenario. The present study uses the opportunities of the asset “Simple Traffic System” and added individual solutions to guarantee such criteria.

With the presented asset, it was possible to equip the virtual environment created in Unity with a traffic system consisting of a waypoint network, traffic light circuits, and the associated vehicles. The interface simplicity of the tool makes it convenient to extend existing static virtual environments with traffic. However, it should be noted that currently, it is only possible to manually create the traffic network with its routes based on waypoints.

The automated creation of the waypoints based on an existing modelled road network or its representation in the form of geospatial data is, therefore, not possible. This circumstance could complicate the implementation of complex city models that have already been created. For example, for 3D city models that were automatically created in Unity based on OpenStreetMap (OSM) and CityGML data [26], developing an automated implementation of the traffic network based on the OSM data could be equipped with the components of the presented asset and could animate the static urban environment with traffic.

It should also be noted that the tool in its current state can only be used to simulate traffic. Virtual tools for analyzing specific traffic parameters, such as volume, pollutant emissions, and noise, have not yet been implemented. Due to the availability of the source code, it is conceivable to develop the asset further and to use it for other tasks that are not only limited to the simulation of traffic.

6. Outlook: The Application in Geography School Education

A highly relevant field for VR applications is education cf. [27,28,29,30]. The presented application has the potential to be implemented in geography education. Together with geography teachers and researchers in the field of didactics, the running traffic system is currently being developed to share knowledge about urban development and smart cities. The traffic system will have different traffic components in the future state of modern cities, which will allow a comparison with the present state. This builds a foundation for geographical discussion in the educational context of secondary schools. The first testing and evaluation of the application was conducted (Figure 7) and organized by the Ministry of Education, North Rhine-Westphalia, Germany. The traffic model also has the potential to be used as a standardized urban scene for laboratory studies focused on the impact of urban traffic transformation on the physiological responses of citizens [31].