Next Article in Journal
On the Stationarity Time of a Vehicle-to-Infrastructure Massive Radio Channel in a Line-of-Sight Suburban Environment
Next Article in Special Issue
A Novel Ionospheric Disturbance Index to Evaluate the Global Effect on BeiDou Navigation Satellite System Signal Caused by the Moderate Geomagnetic Storm on May 12, 2021
Previous Article in Journal
Optimal Water Level Management for Mitigating GHG Emissions through Water-Conserving Irrigation in An Giang Province, Vietnam
Previous Article in Special Issue
Fault-Free Protection Level Equation for CLAS PPP-RTK and Experimental Evaluations
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Performance Evaluation of GNSS Position Augmentation Methods for Autonomous Vehicles in Urban Environments

by
Harihara Bharathy Swaminathan
1,
Aron Sommer
2,
Andreas Becker
3 and
Martin Atzmueller
1,4,*
1
Semantic Information Systems Group, Osnabrück University, 49090 Osnabrück, Germany
2
Aptiv Services Deutschland GmbH, 42119 Wuppertal, Germany
3
Faculty of Information Technology, Dortmund University of Applied Science and Arts, 44139 Dortmund, Germany
4
German Research Center for Artificial Intelligence (DKFI), 49090 Osnabrueck, Germany
*
Author to whom correspondence should be addressed.
Sensors 2022, 22(21), 8419; https://doi.org/10.3390/s22218419
Submission received: 28 September 2022 / Revised: 25 October 2022 / Accepted: 26 October 2022 / Published: 2 November 2022
(This article belongs to the Special Issue Advances in GNSS Positioning and GNSS Remote Sensing)

Abstract

:
Global Navigation Satellite Systems provide autonomous vehicles with precise position information through the process of position augmentation. This paper presents a series of performance tests aimed to compare the position accuracy of augmentation techniques such as classical Differential Global Navigation Satellite System, Real-time Kinematic and Real-time eXtended. The aim is to understand the limitations and choose the best position augmentation technique in order to obtain accurate, trustworthy position estimates of a vehicle in urban environments. The tests are performed in and around the German cities of Wuppertal and Duesseldorf, using a vehicle fitted with the navigation system POS-LV 220, developed by Applanix Corporation. In order to evaluate the real-time performance of position augmentation techniques in a highly challenging environment, a total of four test regions are selected. The four test regions are characterized mainly by uneven terrain with tall buildings around the University of Wuppertal, flat terrain with roads of varying width in the city centre of Wuppertal and Duesseldorf and flat terrain in a tunnel section located in the city of Wuppertal. The performances of the different position augmentation are compared using a Root Mean Square (RMS) error estimate obtained as an output from the Applanix system. Furthermore, a High-Definition map of the environment is used for the purpose of model validation, which justifies the use of RMS error estimate as an evaluation metric for the performance analysis tests. According to the performance tests carried out as per the conditions specified in this paper, the Real-time eXtended (RTX) position augmentation method enables to obtain a more robust position information of the vehicle than Real-time Kinematic (RTK) method, with a typical accuracy of a few centimeter in an urban environment.

1. Introduction

Precise navigation is one of the most important requirements of a fully autonomous car. The car should possess the knowledge of its own position and orientation, in order to navigate precisely within and through an environment [1]. Current road cars use the Global Positioning System (GPS) [2,3,4], an example of the Global Navigation Satellite System (GNSS) which aids the car for estimating its own position. In a navigation system involving standalone GPS, a raw position information signal originates from a geo-stationary satellite and reaches a receiver located on the earth surface. During this phase of travel, pseudo-range errors occur due to the solar radiation in the ionosphere and weather conditions such as clouds, rain and lightning in the troposphere [5]. Even though GPS is used by road cars to estimate their position, its accuracy is only up to a few meters under open sky conditions and the signal is heavily influenced by the aforementioned factors resulting in inaccurate position estimates [6]. As per the Federal Aviation Administration GPS Performance Analysis Report [7], the accuracy of standard GPS is within 3 m, which is insufficient for a fully autonomous car which requires a more precise localization. In contrast to road cars, a fully autonomous car requires an position accuracy of a few centimeter with very high reliability and precision, in order to maintain lane-level accurate localization. Therefore, an improved navigation system is used to estimate the position through augmentation of the raw signal with additional information. One method for improving attributes of a navigation system such as accuracy, reliability and availability, through the integration of external information into the position calculation process is known as Global Navigation Satellite System (GNSS) position augmentation. Some of the widely known methods include Differential Global Navigation Satellite System (DGNSS) [8], Real-time Kinematic (RTK) [9] and Real-time eXtended (RTX) [10].
In this paper, we conduct a series of performance tests aimed to compare the position augmentation accuracy of classical DGNSS, RTK and RTX in the cities such as Wuppertal and Duesseldorf located in Germany. As part of the comparative evaluation, the position augmentation methods are compared using Root Mean Square (RMS) values obtained from the navigation system itself. Our results indicate, that the positional updates obtained using Real-time eXtended process are more robust than Real-time Kinematic with an accuracy of a few centimeter in an urban environment.
Our main contributions are summarized as follows:
  • The experiments in this paper were carried out with a reference base station located few kilometers away as compared to experiments with the reference station located within a kilometer. In order to obtain results indicative of real-time challenges posed to GNSS signal availability, test regions consisting of both flat and uneven terrain posing various challenges to signal availability were selected for analysis.
  • The performance analysis presented in this paper helps in identifying a precise position augmentation method for the purpose of localizing autonomous vehicles in dynamic environments particular to German cities such as Wuppertal and Duesseldorf, characterized by regions consisting of buildings, tunnels and underpasses.

2. Related Work

Until now, Scherzinger [11] presented the performance of Position and Orientation System for Land Vehicles (POS LV) with Inertially Aided Real-time Kinematic position augmentation system (IARTK) through one of the tests in downtown Tokyo to study the behavior of Applanix POS LV system during partial and total GPS Outages. In their paper, a tightly coupled inertial-GPS integration was introduced for obtaining inertially aided RTK position updates. Through this way of implementation, the navigation solution was able to provide position updates even during phases when fewer than four satellites were visible. Furthermore, the tightly coupled inertial-GPS integration enabled position updates with an accuracy of several meters during partial and total GPS outage situations.
Zywiel [12] described the performance results of a test where a van equipped with an Applanix POS LV aided inertial navigation system was driven through a suburban area located north of Toronto, Canada. Frank and Louis [13] presented the results of a test carried out in downtown Toronto to determine the operational capabilities of the Applanix POS LV 200 system. In their paper, the position accuracy during the test is presented using Root Mean Square (RMS) values obtained directly as output from the POS LV 200, which we have also followed for performance analysis presented in the later sections of this paper. The aforementioned works established the ability of aided inertial systems to provide a continuous, accurate position information in urban environments.
Erol et al. [14] presented a performance analysis of real-time position coordinates obtained through the commercial service, the Trimble CenterPoint RTX in a maritime environment. In an experiment carried out on a lake dam in Turkey, the Trimble CenterPoint RTX was capable of achieving centimeter level accuracy. This study implied that a real-time position service such as the CenterPoint RTX is a powerful tool for obatining highly accurate position even for dynamic applications. Ng et al. [15] evaluated the performance of RTK in land area that is obstructed by buildings and in uneven landscape. It was evident that the position accuracy of RTK was affected due to factors such as uneven landscape and blockage of signal from the base station by buildings. Overall, to the best of authors’ knowledge, an analysis comparing the accuracy of various GNSS position augmentation methods such as the DGNSS, RTK and RTX with focus on a dynamic urban environment has not yet been performed.
The rest of this paper is organized as follows. Section 3 gives an overview about Global Navigation Satellite System and Position augmentation methods such as RTK and RTX. Then, Section 4 introduces main components of the vehicle used for data collection and evaluation metrics. After that, Section 5 discusses the results of performance analysis. Finally, Section 6 concludes the paper with a summary and outlines interesting directions for future research.

3. Background: An Overview on the Global Navigation Satellite System

Global Navigation Satellite System is a term that is broadly used to refer all satellite-based systems comprising a constellation of satellites orbiting over the earth’s surface and continuously transmitting signals enabling users to determine their position on earth. One universally well-known system of GNSS is the Global Positioning System, developed by the US Department of Defence for public use. Other systems of GNSS currently include the Russian Global Navigation Satellite System (GLONASS), the European satellite system (Galileo) and China’s Beidou. Furthermore, regional satellite navigation systems such as Quasi-Zenith Satellite System (QZSS) operated by Japan and Indian Regional Navigation Satellite System (IRNSS) are also part of GNSS [16]. A GNSS-compatible hardware equipment is capable of using navigational satellites from networks beyond the GPS system for increased position accuracy and reliability.
Each GNSS, like the GPS, comprises of three major components such as the space segment, control segment and user segment. The space segment consists of GNSS satellites arranged in a constellation, orbiting about 20,000 Km above the earth’s surface. Each satellite transmits two carrier waves, referred to as L1 and L2 with information related to time, the satellite’s orbital position and status. The control segment is made a ground-based network of master control stations, data uploading stations and monitor stations. The GPS comprises of two master control stations, four data uploading stations and 16 monitor stations located throughout the world. In a GNSS system, the master control station adjusts the satellite position information and onboard high-precision clocks when necessary to maintain accuracy. The monitor stations installed over a broad geographical area monitor the satellite signals and status and relay this information to the master control station. The master control station analyses the signals and then transmits position and time corrections to the satellites through data uploading stations. The user segment consists of equipment that processes the received signals from the GNSS satellites and uses them to derive location and time information. The equipment used in the user segment could be a smartphone or an improved receiver for survey and mapping purposes by an autonomous vehicle. Each of this equipment has one or two antennas to receive either one or both carrier signals sent by a satellite depending upon the accuracy requirements and a receiver to process the radio signals, calculate position and time [17]. A standalone GNSS service such as the GPS is accurate only within a few meters and does not satisfy the position requirements of a fully autonomous vehicle as it is susceptible to a wide range of errors, namely the time delay introduced by the layers of the atmosphere, the ionosphere and troposphere, effects such as multipath due to reflections from nearby objects. In addition, errors introduced on satellite clocks and orbit information also result in inaccurate position. Therefore, techniques and equipment have been developed to improve the accuracy and availability of GNSS position and time information. In the remaining part of this section, techniques introduced to improve the accuracy and availability of GNSS are explained.
GNSS systems can be broadly classified into code-based and carrier phase-based measurement according to the technique used by the receiver to calculate the travel time of the satellite signal. A receiver determines the time by comparing a pseudo random code generated by itself, with an identical code in the signal. In case of GPS, the pseudo random code is referred as the coarse acquisition C/A code which is freely available to public users. This process of matching codes to estimate the travel time of a signal is termed as code-based measurement and is highly prone to errors. On the other hand, the phase-based measurement technique is performed by the receiver estimating phase differences between L1, L2 carrier waves to achieve millimeter level precision. Furthermore, GNSS techniques are also classified into absolute and differential depending upon the nature in which the receiver determines position. In absolute GNSS, a modeling of random error sources are carried out to calculate the position. On the other hand, a differential GNSS involves the use of correction data from a base station whose position is known to a high degree of accuracy for the calculation of position. The overall classification of GNSS systems is shown in Figure 1.
A Space Based Augmentation System (SBAS) [18,19,20] uses a network of reference base stations on the ground to obtain positional error data for a particular area. Subsequently, this information is sent up to a geo-stationary satellite before being broadcasted back down to all receivers in a particular area. In comparison to SBAS, a Ground Based Augmentation System (GBAS) [21,22], is a complete on-ground facility to provide corrections. Both of these systems are accurate up to 1 m. Real-time Kinematic (RTK) is a technique widely used to receive precise position information using corrections that are transmitted via some sort of a data communication (e.g., 3G/LTE connection) from a single base or multiple reference base stations with an accuracy of less than 2 cm. The main drawback of the single base RTK technique is the necessity of a reference station located close to user. For a large area, the RTK is extended from a single base to a multi-base technique. The major success of RTK is attributed to the development of Network RTK which resulted in a tremendous reduction of the investment costs necessary to start an RTK service, since the number of reference stations can be reduced [23]. In case of RTK, the availability of corrections is dependent on the prevailing environmental conditions. For example, a rugged terrain condition could possibly interrupt the correction signal, even if the distance between the base station and the receiver is short [14]. Some providers of RTK services are SAPOS [24] with over 270 stations in Germany and Trimble VRS Now [25] with 205 stations throughout Europe. In the Precise Point Positioning (PPP) method, the data collected with a single GNSS receiver is used with precise satellite orbit and clock information from data analysis centres such as the International GNSS Service (IGS) to achieve centimeter level position accuracy. However, the requirement of relatively long convergence time to achieve the cm-level precision is one of the main drawbacks of this method, thereby limiting the use of PPP on real-time applications [14]. The Trimble CenterPoint Real-time eXtended (RTX) belonging to Real-time PPP (RT-PPP) method is a type of GNSS correction service that provide position accuracy of a few centimeter through an L-band satellite communication or Internet. In contrast to RTK, RTX does not require the local reference station as Satellite orbits, satellite clock offsets and atmospheric errors are estimated in real-time for any place on the globe using innovative and advanced data processing algorithms by analysis centres.

4. Method

In this section, the vehicle setup and the evaluation metric used for the comparison of GNSS position augmentation methods is described.

4.1. Vehicle Setup

The Applanix POS LV-220 system, developed and manufactured by Applanix: A Trimble Company based in Richmond Hill, Ontario, Canada, is mounted onto the test vehicle consisting mainly of an inertial measurement unit (IMU), a GNSS signal receiver, an Inertial navigator component and a error controller component consisting of a Kalman filter and an Error controller [26]. A block diagram consisting of all the components within an integrated inertial system such as the Applanix POS LV-220 is shown in Figure 2. In the Applanix POS LV-220, the GNSS signal receiver is configured to receive position updates such as RTK and RTX through a cellular network. Due to the fact that a combination of GNSS techniques to obtain position and IMU is used to generate accurate, continuous position and orientation information, the Applanix POS LV-220 is termed as an integrated inertial system. The IMU comprises of accelerometers and gyroscopes to measure the 3D accelerations and orientations. Even though a complete navigation solution can be provided by using IMU without any external information, the position and orientation errors grow unbounded over time in such a free-inertial system due to bias errors [27]. This drawback of a free-inertial system is overcome by using GNSS position updates to aid the IMU and thereby decrease the position and orientation errors. Therefore, the Applanix POS LV-220 system consists of a GNSS receiver to receive positional updates through GNSS position augmentation methods mentioned in the earlier section. The inertial navigator component receives accelerations and angular rates from the IMU’s accelerometers and gyros to compute the IMU position, velocity and three-axis attitude such as roll, pitch and yaw. Since the computed components of the inertial navigator are prone to error such as initialization and inertial sensor errors, an Error controller component is used for controlling these errors by adjusting the Inertial navigator integration process and calibrating the gyro and accelerometer. The error regulation component comprises the Kalman filter and the closed-loop error controller. The Kalman filter maintains a multivariable linear stochastic model of the inertial navigator errors as mentioned in [28]. The Kalman filter is a recursive least squares estimator that computes optimal estimates of errors from measurement of position and velocity computed by the inertial navigator differenced with corresponding position and velocity from the embedded GNSS receiver. By estimating the accelerometer and gyro biases, the Kalman filter effectively calibrates the inertial sensors dynamically during normal operation. The error controller computes resets to the inertial navigator integration processes based on the Kalman filter estimates, and applies these so as to cause the estimated errors to be nulled. Due to this error control loop, the initial alignment of the inertial navigator and subsequent continuous error regulation in all navigation quantities are achieved. As the errors in the inertial navigation component are continuously and optimally controlled by the error regulation component, this integration is termed as “tightly-coupled” [29]. As an output from the Applanix POS LV-220 system, the position information consisting of latitude, longitude and height, the heading, roll and pitch of the vehicle is available at all times.

4.2. Evaluation Metric

In the Applanix POS LV-220 system, the performance metrics are expressed in meters with North RMS error, East RMS error and Down RMS error corresponding to Northing, Easting and Altitude coordinates, respectively. All these RMS errors are scaled estimates provided by the navigation system based on prevailing conditions such as the number of satellites visible, the quality of signal, etc. influencing the accuracy of position coordinates. In this research, a single RMS value calculated by averaging the North RMS error and East RMS error is used to denote positional accuracy. Therefore, an RMS value close to zero is indicative of precise position than a much higher value.

5. Results

A total of four different test regions were selected based on varying levels of traffic congestion, challenges to signal availability due to the presence of tall buildings and trees in the region. The path driven by the vehicle in these test regions are shown in Figure 3. The test region shown in Figure 3a is located around the University of Wuppertal and is characterized by few tall buildings, an underpass and also a residential area. A single trip around this test region covers a distance of 1.89 km with an elevation gain/loss of ±50 m and an average slope of 4.4%. The test region 2 shown in Figure 3b is located in the city centre of Wuppertal. This region is characterized with narrow road sections and highly dense traffic. A lap around the test region covers a distance of 3.43 km with an elevation gain/loss of ±63 m and an average slope of 3.1%. The test region 3 shown in Figure 3c is located in the city centre of Duesseldorf. This region comprises of streets with medium and highly congested traffic. A single lap around this test region covers a distance of 7.88 km with an elevation gain/loss of ±86 m and an average slope of 1.6%. Finally, the test region 4 shown in the Figure 3d comprises of the Burgholz tunnel which is 1865 m long and facilitates a direct and continuous expressway connection between the German motorways (Autobahn) A1, A46.
It is essential to carry out model validation to ensure whether large RMS values are indicative of position degradation as these values are estimated by the Applanix solution, being used for performance evaluation of position augmentation methods in this paper. For this purpose, a small section of the test-region is selected in such a way that the test vehicle can be localized approximately. Due to the absence of ground truth, it was ensured that the vehicle was driven close to the outer lane boundary in this test region as shown in Figure 4. For the purpose of localizing the vehicle laterally between the inner boundary and outer boundary of the lane section, the width of test vehicle is considered. For this approximation, the lane dimensions were obtained from an HD map with position accuracy of less than 2 cm, provided by 3D Mapping Solutions GmbH, Holzkirchen, Germany [30].
In Figure 5a, the reconstructed path of the test vehicle (in yellow), the lane dimensions obtained from the HD map (in Red) and the estimated path of the test-vehicle using DGNSS position updates (in Blue) are plotted. Similarly, Figure 5b comprises of the reconstructed path of the test vehicle, the lane dimensions obtained from the HD map and the estimated path of the test-vehicle using RTX position updates. In order to determine the error at an estimated position on the test-region, a metric termed as Absolute error is used. The Absolute error for every position (p) on the test-region estimated by the Applanix solution can be determined by finding the nearest point (q) on the approximated position of the test vehicle based on the euclidean distance. Each position (p,q) in the test-region is assigned a Universal Transverse Mercator (UTM) coordinate as the UTM is a system for assigning coordinates to locations on the surface of the earth. According to the UTM system, each position on the surface of earth is assigned an Easting (m), a Northing (m), a Down (m) and a UTM zone value. The Absolute Error is the mean of two individual position error components: the Easting Error and a Northing Error. For every position estimate obtained from the Applanix solution, the individual error components and the overall Absolute Error can be determined in the following way:
E r r o r e a s t i n g = q E a s t i n g p E a s t i n g
E r r o r n o r t h i n g = q N o r t h i n g p N o r t h i n g
A b s o l u t e E r r o r = E r r o r e a s t i n g + E r r o r n o r t h i n g 2
In Figure 5c,d, violin plots showing the distribution of Mean RMS values estimated from the Applanix solution and the calculated Absolute Error for the selected region are provided. In a violin plot, a kernel density estimation is used to represent the distribution shape of data on either sides of the thin grey line. In particular, the wider sections are used to represent a higher probability that members of the population will take on the given value and the skinnier sections represent a lower probability. The thick grey bar in each violin represents the interquartile range and thin grey line indicates rest of point distribution excluding the outliers. Finally, the white dot represents the median value. From the violin plots, it can be observed that high Mean RMS values are indicative of high position error as exhibited by DGNSS and low Mean RMS values are indicative of better positioning performance as exhibited by RTX. As a result of this model validation, the mean RMS values estimated by the Applanix solution are henceforth used as an evaluation metric for the performance analysis across all the test-regions.
In order to compare the positional accuracy across various position augmentation techniques such as DGNSS, RTK and RTX in the selected test regions, horizontal violin plots showing the distribution of RMS values are used. Figure 6 consists of these horizontal violin plots that depict the positional accuracy of the augmentation techniques across the selected test regions. In particular, Figure 6a depicts the test region around the University of Wuppertal, Figure 6b depicts the test region located in the city centre of Wuppertal, Figure 6c depicts the test region located in the city centre of Duesseldorf and Figure 6d depicts the test region consisting of the Burgholz tunnel in Wuppertal.
It is observed from the violin plots of test regions shown in Figure 6a–c that the RMS values of DGNSS-based position augmentation are far higher than the values estimated with techniques such as RTK or RTX. The fluctuation of positional accuracy with DGNSS, particularly in a complete urban environment such as the city centre of Duesseldorf, Wuppertal is observed with the length of thin grey line in Figure 6b,c. The aforementioned behaviour of DGNSS is due to the use of an outdated protocol. On the other hand, RTK and RTX uses sophisticated protocols such as the CMR, CMR+ or RTCM 3 to perform error calculation for correcting the ionospheric delays and satellite clock errors. Furthermore, the deterioration of RTX and RTK correction signals inside the Burgholz tunnel resulted in the Applanix system switching to a less accurate position augmentation modes. This can be observed in Figure 6d, where the range of RMS values for the performance of RTK and RTX is higher than the values observed for other test regions. The range of RMS values for the performance of RTK and RTX is higher in the test region involving the Burgholz tunnel because the tunnel is preventing the GNSS receiver from receiving any correction or positioning signal termed as GNSS outage. As a result of this, the errors in the IMU accumulate over time.
A detailed percentage ratio of data samples acquired during the performance test carried out in the test region consisting of the Burgholz tunnel is shown in the Table 1 and Table 2. In particular, Table 1 shows the percentage distribution of data samples acquired during the laps with RTK position augmentation. During the course of four laps driven through the Burgholz tunnel with RTK position augmentation, the majority of samples were acquired from methods with accuracy close to 75 m and 1 m.
In a similar way, Table 2 shows the percentage distribution of data samples acquired during the laps with RTX position augmentation. The percentage distribution of data samples observed during these laps is similar to the behavior observed during RTK position augmentation with the majority of samples acquired from methods with accuracy close to 75 m and 1 m.
The expected accuracy of position augmentation methods given in the column names of Table 1 and Table 2 were provided by Applanix corporation in the interface document for POS LV system. The description of position methods whose accuracy is provided as column names in the previous tables is given below in the Table 3:
For the sake of comparison between RTX and RTK in detail, the laps recorded around the University of Wuppertal are considered. The RMS values for the laps are shown in Figure 7a and Figure 7b respectively. In these figures, the location of a peak RMS values is visible across all the laps especially in between samples 10,000 and 15,000. The peak RMS values observed in this interval is due to the presence of tall buildings and underpasses during the course of a lap. The presence of peak RMS values in the areas with tall buildings and underpasses indicate the deterioration of RTK/RTX position updates. In Figure 7b, several smaller RMS peaks are observed followed by minimum values between samples 15,000 and 25,000 indicating that the navigation system is able to re-initialize with minimal position errors after a short duration of RTX signal outage. This ability of RTX position updates to re-initialize the navigation system with minimum errors can also be verified in Figure 6a violin plot as multi-modal distribution of RMS values indicated using two wider sections. In contrast to RTX, the RMS values estimated during RTK position updates in Figure 6a violin plot are uni-modal in nature.
In order to understand the impact of vehicle speed on positioning inaccuracies, the laps recorded aroud the University of Wuppertal are considered again. For this purpose, the vehicle speed for the laps involving RTK and RTX position updates shown in Figure 8a,b are compared against the Mean RMS value plots in Figure 7a and Figure 7b respectively. As general observation for both RTK and RTX updates, the position accuracy is better, i.e., lower Mean RMS values when the vehicle is traveling at low speed or is stationary particularly between samples 15,000 and 20,000. This can be attributed to the fact that GNSS receivers are able to receive position corrections at correct times during lower speeds as compared against higher speeds. As the main objective of the performance evaluation presented in this paper is to analyze various augmentation methods by taking into consideration real-life factors such as the maximum permitted speed in the test region, the dynamic change in traffic situations and various other challenges posed to signal availability, detailed experiments involving different speeds were not performed.

6. Conclusions

In this paper, we presented performance tests conducted to compare the position augmentation accuracy of classical DGNSS, RTK and RTX in the cities of Wuppertal and Duesseldorf in Germany. The four test regions are characterized mainly by uneven terrain with tall buildings around the University of Wuppertal, flat terrain with roads of varying width in the city centre of Wuppertal and Duesseldorf and flat terrain in a tunnel section located in the city of Wuppertal. For the performance comparison between the position augmentation techniques, we used horizontal violin plots in order to visualize the distribution of the respective Root Mean Square error values.
As a general observation, the navigation system switched to an available position augmentation method during phases of RTK/RTX signal outages. In particular, in the test region consisting of a tunnel section, the navigation system also switched to a less precise code-based GNSS position augmentation method, minutes after the vehicle entered into this tunnel section. Furthermore, position augmentation services such as RTK and RTX outperformed the classical DGNSS in the test regions with partial to complete open sky conditions. This phenomenon can be observed in the violin plots depicting the distribution of the Root Mean Square values across various position augmentation techniques for each test region.
Also, the Root Mean Square values of DGNSS-based position augmentation are higher than the estimated values with techniques such as RTK and RTX. From the violin plots, the fluctuation of positional accuracy with DGNSS is also observed in an urban environment such as the city centre of Duesseldorf and Wuppertal. Even though the performance of RTK and RTX correction signals deteriorated in a test area consisting of an underpass around the University of Wuppertal, the effectiveness of RTX position updates is seen when the navigation system re-initializes itself after signal outage with minimum position error. As a conclusion from the performance analysis, our results indicate that the RTX augmentation method is proven to be precise and robust even within the urban centres of Duesseldorf and Wuppertal.
In the future, the need for robust position estimation algorithms in autonomous vehicles is inevitable as the results from show that the GNSS position augmentation services such as the classical DGNSS, RTX and RTK are highly affected by the condition of a test region. Therefore, robust position estimation algorithms play an important role especially in areas of complete signal outage using a rough position estimate obtained from the navigation system, a map of the surrounding and scan from an environment perception sensor like an autonomous radar, camera or lidar.

Author Contributions

Conceptualization, H.B.S. and A.S.; Data curation, H.B.S.; Formal analysis, H.B.S.; Funding acquisition, A.S.; Investigation, H.B.S.; Methodology, H.B.S., A.S., A.B. and M.A.; Visualization, H.B.S.; Writing—original draft, H.B.S.; Writing—review and editing, A.S., A.B. and M.A. All authors have read and agreed to the published version of the manuscript.

Funding

This research has been partially supported by Aptiv Services Deutschland GmbH, 42119 Wuppertal, Germany.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Restrictions apply to the availability of these data. Data was obtained from Aptiv Services Deutschland GmbH and are not available publicly.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Pishehvari, A.; Iurgel, U.; Stefer, M.; Tibken, B. Range-Doppler Registration. In Proceedings of the 5th International Conference on Control and Robotics Engineering (ICCRE), Osaka, Japan, 24–26 April 2020; pp. 169–176. [Google Scholar]
  2. Parkinson, B.W.; Gilbert, S.W. NAVSTAR: Global positioning system—Ten years later. Proc. IEEE 1983, 71, 1177–1186. [Google Scholar] [CrossRef]
  3. Parkinson, B.W. Global Positioning System: Theory and Applications II. Prog. Astronaut. Aeronaut. 1996, 164, 3–50. [Google Scholar]
  4. Enge, P.K. The global positioning system: Signals, measurements, and performance. Int. J. Wirel. Inf. Netw. 1994, 1, 83–105. [Google Scholar] [CrossRef]
  5. Hornbostel, A. Propagation Problems in Satellite Navigation. In Proceedings of WFMN07, Chemnitz, Germany, June 2009; pp. 21–30. [Google Scholar]
  6. Kuutti, S.; Fallah, S.; Katsaros, K.; Dianati, M.; Mccullough, F.; Mouzakitis, A. A survey of the state-of-the-art localization techniques and their potentials for autonomous vehicle applications. IEEE Internet Things J. 2018, 5, 829–846. [Google Scholar] [CrossRef]
  7. Team, GPS Product. Global Positioning System (GPS) Standard Positioning Service (SPS) Performance Analysis Report; GPS Product Team: Washington, DC, USA, 2014. [Google Scholar]
  8. Walsh, D.; Capaccio, S.; Lowe, D.; Daly, P.; Shardlow, P.; Johnston, G. Real time differential GPS and GLONASS vehicle positioning in urban areas. Space Commun. 1997, 14, 203–217. [Google Scholar]
  9. Riley, S.; Talbot, N.; Kirk, G. A new system for RTK performance evaluation. In Proceedings of the Position Location and Navigation Symposium, San Diego, CA, USA, 13–16 March 2000; pp. 231–236. [Google Scholar]
  10. Leandro, R.; Landau, H. RTX positioning: The next generation of cm-accurate real-time GNSS positioning. In Proceedings of the 24th International Technical Meeting of The Satellite Division of the Institute of Navigation (ION GNSS 2011), Portland, OR, USA, 19–23 September 2011; pp. 1460–1475. [Google Scholar]
  11. Scherzinger, B.M. Inertially aided RTK performance evaluation. In Proceedings of the 15th International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GPS 2002), Portland, OR, USA, 24–27 September 2002; pp. 1429–1433. [Google Scholar]
  12. Zywiel, M.T. Aided inertial technology: Positioning for the future. Traffic Technol. Int. XP 2002, 2673726, 1–3. [Google Scholar]
  13. Artes, F.; Nastro, L. Applanix POS LV 200 Generating Continuous Positioning Accuracy in the Absence of GPS Reception; Applanix: Richmond Hill, ON, Canada, 2005. [Google Scholar]
  14. Erol, S.; Alkan, R.M.; Ozulu, İ.M.; İlçi, V. Performance analysis of real-time and post-mission kinematic precise point positioning in marine environments. Geod. Geodyn. 2020, 11, 401–410. [Google Scholar] [CrossRef]
  15. Ng, K.M.; Johari, J.; Abdullah, S.A.C.; Ahmad, A.; Laja, B.N. Performance evaluation of the RTK-GNSS navigating under different landscape. In Proceedings of the 2018 18th International Conference on Control, Automation and Systems (ICCAS), PyeongChang, Korea, 17–20 October 2018; pp. 1424–1428. [Google Scholar]
  16. Hegarty, C.J. GNSS signals—An overview. In Proceedings of the 2012 IEEE International Frequency Control Symposium, Baltimore, MD, USA, 21–24 May 2012; pp. 1–7. [Google Scholar]
  17. McNeff, J.G. The global positioning system. IEEE Trans. Microw. Theory Tech. 2002, 50, 645–652. [Google Scholar] [CrossRef]
  18. Enge, P.; Walter, T.; Pullen, S.; Kee, C.; Chao, Y.; Tsai, Y. Wide area augmentation of the global positioning system. Proc. IEEE 1996, 84, 1063–1088. [Google Scholar] [CrossRef]
  19. Hegarty, C.J.; Chatre, E. Evolution of the global navigation satellitesystem (gnss). Proc. IEEE 2008, 96, 1902–1917. [Google Scholar] [CrossRef]
  20. Walter, T. Satellite-Based Augmentation Systems (SBASs). In Position, Navigation, and Timing Technologies in the 21st Century: Integrated Satellite Navigation, Sensor Systems, and Civil Applications; John Wiley & Sons.: New York, NY, USA, 2020; Volume 1, pp. 277–306. [Google Scholar]
  21. Murphy, T.; Imrich, T. Implementation and operational use of Ground-Based Augmentation Systems (GBASs)—A component of the future air traffic management system. Proc. IEEE 2008, 96, 1936–1957. [Google Scholar] [CrossRef]
  22. Crosby, G.K.; Kraus, D.K.; Ely, W.S.; Cashin, T.P.; McPherson, K.W.; Bean, K.W.; Stewart, J.M.; Elrod, B.D. A Ground-based Regional Augmentation System (GRAS)-The Australian Proposal. In Proceedings of the 13th International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GPS 2000), Salt Lake City, UT, USA, 19–22 September 2000; pp. 713–721. [Google Scholar]
  23. Vollath, U.; Landau, H.; Chen, X. Network RTK–concept and performance. In Proceedings of the GNSS Symposium, Wuhan, China, 2002; pp. 27–30. [Google Scholar]
  24. Satellitenpositionierungsdienst SAPOS. Available online: https://www.bezreg-koeln.nrw.de/brk_internet/geobasis/raumbezug/sapos/index.html (accessed on 19 May 2021).
  25. VRS Now. Available online: https://positioningservices.trimble.com/services/vrs/vrs-now/ (accessed on 19 May 2021).
  26. Whittaker, W.; Nastro, L.T. Get Ready, Get Set, Race!: Position and Orientation Data for Autonomous Navigation. GPS World 2006, 17, 42–47. [Google Scholar]
  27. Artes, F.; Hutton, J.T. GPS and Inertial Navigation Delivering. GEOconnexion Int. Mag. 2005, 52–53. [Google Scholar]
  28. Scherzinger, B.M. Inertial navigator error models for large heading uncertainty. In Proceedings of Position, Location and Navigation Symposium-PLANS’96, Atlanta, GA, USA, 22–25 April 1996; pp. 477–484. [Google Scholar]
  29. Woolven, S.; Scherzinger, B.; Field, M. POS/MV-system performance with inertial/RTK GPS integration. In Proceedings of the Oceans’ 97. MTS/IEEE Conference, Halifax, NS, Canada, 6–9 October 1997; Volume 2, pp. 1104–1108. [Google Scholar]
  30. 3D Mapping Solutions GmbH. Available online: https://www.3d-mapping.de/ (accessed on 31 October 2022).
Figure 1. Classification of GNSS systems.
Figure 1. Classification of GNSS systems.
Sensors 22 08419 g001
Figure 2. Block diagram of components within an integrated inertial system such as the Applanix POS LV-220 [13].
Figure 2. Block diagram of components within an integrated inertial system such as the Applanix POS LV-220 [13].
Sensors 22 08419 g002
Figure 3. The selected test regions for performance analysis.
Figure 3. The selected test regions for performance analysis.
Sensors 22 08419 g003
Figure 4. Approximation of vehicle position (in Yellow) in between lanes using lane dimensions of the selected test-region obtained from HD map and vehicle width dimension.
Figure 4. Approximation of vehicle position (in Yellow) in between lanes using lane dimensions of the selected test-region obtained from HD map and vehicle width dimension.
Sensors 22 08419 g004
Figure 5. Model validation to establish relationship between high RMS values and position degradation. (a) Reconstructed Vs Estimated path using DGNSS position updates of the test-vehicle. (b) Reconstructed Vs Estimated path using RTX position updates of the test-vehicle. (c) Mean RMS as derived from Applanix POS LV-220 system. (d) Absolute Error as calculated by using reconstruction.
Figure 5. Model validation to establish relationship between high RMS values and position degradation. (a) Reconstructed Vs Estimated path using DGNSS position updates of the test-vehicle. (b) Reconstructed Vs Estimated path using RTX position updates of the test-vehicle. (c) Mean RMS as derived from Applanix POS LV-220 system. (d) Absolute Error as calculated by using reconstruction.
Sensors 22 08419 g005
Figure 6. Horizontal violin plots showing the distribution of RMS values across various position augmentation techniques for each test region. The white dot represents the median RMS error value, the thick grey bar in the center represents the interquartile range and the thin grey line represents the rest of the distribution.
Figure 6. Horizontal violin plots showing the distribution of RMS values across various position augmentation techniques for each test region. The white dot represents the median RMS error value, the thick grey bar in the center represents the interquartile range and the thin grey line represents the rest of the distribution.
Sensors 22 08419 g006
Figure 7. RMS values estimated by the Applanix POS LV-220 system during the course of laps around the University of Wuppertal. (a) Estimated RMS values during RTK position updates. (b) Estimated RMS values during RTX position updates.
Figure 7. RMS values estimated by the Applanix POS LV-220 system during the course of laps around the University of Wuppertal. (a) Estimated RMS values during RTK position updates. (b) Estimated RMS values during RTX position updates.
Sensors 22 08419 g007
Figure 8. Speed of the vehicle for laps around the University of Wuppertal.
Figure 8. Speed of the vehicle for laps around the University of Wuppertal.
Sensors 22 08419 g008
Table 1. Percentage ratio of data samples and the corresponding expected accuracy during the laps with RTK position augmentation through the Burgholz tunnel.
Table 1. Percentage ratio of data samples and the corresponding expected accuracy during the laps with RTK position augmentation through the Burgholz tunnel.
Laps75 m1 m0.25 m0.02 m
186.228.794.980.00
222.2769.857.860.00
330.3118.8716.1934.63
427.0861.9310.980.00
Table 2. Percentage ratio of data samples and the corresponding expected accuracy during the laps with RTX position augmentation through the Burgholz tunnel.
Table 2. Percentage ratio of data samples and the corresponding expected accuracy during the laps with RTX position augmentation through the Burgholz tunnel.
Laps75 m1 m0.05 m
182.2015.542.28
253.5144.262.23
355.0142.842.15
441.7548.2210.03
Table 3. The Description of position methods whose expected accuracy is provided as column names in Table 1 and Table 2.
Table 3. The Description of position methods whose expected accuracy is provided as column names in Table 1 and Table 2.
DescriptionExpected Accuracy
C/A mode75 m
3-dimension DGNSS mode1 m
Float RTK mode0.25 m
Integer narrow lane RTK mode0.02 m
Trimble RTX Mode0.05 m
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Swaminathan, H.B.; Sommer, A.; Becker, A.; Atzmueller, M. Performance Evaluation of GNSS Position Augmentation Methods for Autonomous Vehicles in Urban Environments. Sensors 2022, 22, 8419. https://doi.org/10.3390/s22218419

AMA Style

Swaminathan HB, Sommer A, Becker A, Atzmueller M. Performance Evaluation of GNSS Position Augmentation Methods for Autonomous Vehicles in Urban Environments. Sensors. 2022; 22(21):8419. https://doi.org/10.3390/s22218419

Chicago/Turabian Style

Swaminathan, Harihara Bharathy, Aron Sommer, Andreas Becker, and Martin Atzmueller. 2022. "Performance Evaluation of GNSS Position Augmentation Methods for Autonomous Vehicles in Urban Environments" Sensors 22, no. 21: 8419. https://doi.org/10.3390/s22218419

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop