Elsevier

Expert Systems with Applications

Volume 42, Issue 3, 15 February 2015, Pages 1538-1550
Expert Systems with Applications

Review
A review on computational intelligence methods for controlling traffic signal timing

https://doi.org/10.1016/j.eswa.2014.09.003Get rights and content

Highlights

  • We have a review of applying machine learning methods (ML) for traffic signal timing.

  • Signal timing through ML methods for a single intersection.

  • Using genetic algorithm to determine NN and FLS parameters.

  • Showing the better performance of ML methods compared to the pre-defined method.

  • Showing better performance of proposed NN and FLS controllers compared to QL.

Abstract

Urban traffic as one of the most important challenges in modern city life needs practically effective and efficient solutions. Artificial intelligence methods have gained popularity for optimal traffic light control. In this paper, a review of most important works in the field of controlling traffic signal timing, in particular studies focusing on Q-learning, neural network, and fuzzy logic system are presented. As per existing literature, the intelligent methods show a higher performance compared to traditional controlling methods. However, a study that compares the performance of different learning methods is not published yet. In this paper, the aforementioned computational intelligence methods and a fixed-time method are implemented to set signals times and minimize total delays for an isolated intersection. These methods are developed and compared on a same platform. The intersection is treated as an intelligent agent that learns to propose an appropriate green time for each phase. The appropriate green time for all the intelligent controllers are estimated based on the received traffic information. A comprehensive comparison is made between the performance of Q-learning, neural network, and fuzzy logic system controller for two different scenarios. The three intelligent learning controllers present close performances with multiple replication orders in two scenarios. On average Q-learning has 66%, neural network 71%, and fuzzy logic has 74% higher performance compared to the fixed-time controller.

Introduction

The increasing volume of traffic in cities has a significant effect on the road traffic congestion and therefore the time it takes for road users to reach their destination. Widening roads and increasing their capacity is not sufficient by itself, as the intersection then become a bottleneck. Bottlenecks cannot be prevented, however, the way intersections are controlled has room for improvement.

Since the 1960’s different methods have been presented to manage intersections and for controlling traffic signals’ timing. As one of the first traffic signal controllers, fixed-time or pre-timed controllers applied historical data to determine appropriate time for traffic signals (Cai, 2010). Fixed-time method is not based on current traffic demands and cannot handle unexpected conditions in traffic. Based on the historical traffic volume, the cycle time is divided to several phases. A fixed amount of time is required for clearing the intersection and starting the next phase after each phase, called the safety time. The safety time increases per hour for the case of shorter cycle time. Therefore, there is a lower overall capacity for intersections with shorter cycle times. On the other hand, longer waiting times and longer queues are the consequences of longer cycle times. Webster proposed a formula based on the flow rate of each lane in a link to solve this issue. This formula is useful to find an optimal cycle and appropriate duration for green time in each phase (Webster, 1958). As fixed-time methods cannot predict traffic demand accurately, they are not appropriate for situations such as accidents, and other disturbances that may disrupt traffic conditions.

The next step for improving the control method was actuated or real time controllers. This type of controller emerged in the 1970’s. Traffic-actuated control methods utilize inductive detectors to observe the actual traffic situation. The traffic-actuated controller must have the ability to determine whether the last vehicle of the queue formed at the stop line during the red phase has passed. This detection is useful for having efficient extension or termination of green time, and it is performed by measuring the gap between vehicles. The green time is terminated when the gap between vehicles is larger than the threshold maximum gap. The optimal placement of detectors at an intersection impacts the performance of actuated method. In addition, by increasing the number of detectors the accuracy of the system is improved. In actuated methods, a pre-specified block period time is considered for extending the green time of a phase. Therefore, detection of sparse traffic can have a considerable influence on delay time (Koonce et al., 2008). MOVA (Vincent & Peirce, 1988), LHVORA (Kronborg & Davidsson, 1993), and SOS (Kronborg & Davidsson, 1996) are samples of actuated traffic control system.

Parameters like time, day, season, weather, and some unpredictable situations such as accidents and special events are highly influential on traffic load. Traffic-adaptive control systems was created to take these elements in account in order to more efficiently predict green times. Fixed-time and actuated method do not use a control policy or a parameterized value function. Furthermore, these systems do not utilize accumulative information for improving their performances. In adaptive traffic control systems, the traffic condition is sensed and monitored continuously and the timing of traffic signals is adjusted accordingly. It is useful to note that adaptive controllers and real-time ones are two different concepts, however, it is possible to have a system with both abilities. The controllers with real-time ability in response to sensory inputs are real-time systems, in which the parameters of the controller and internal logic remain unchanged. Alternatively, one of the special features of adaptive systems is their characteristic in adjusting their parameters and internal logic in response to the significant change of the environment (Abdulhai, Pringle, & Karakoulas, 2003).

Both SCATS (Sims & Dobinson, 1980) and SCOOTS (Hunt, Robertson, & Bretherton, 1982) are famous adaptive systems that gather data of the traffic flow in real-time at each intersection to control timing of traffic lights. To obtain traffic information SCATS counts vehicles at each stop line, and SCOOTS applies a set of advanced detectors placed upstream of the stop line. Using these detectors, SCOOTS gives a higher resolution of the traffic condition such as traffic flow and number of cars in the queue before they reach the stop line. SCATS and SCOOTS both use centralized control. OPAC (Gartner, Tarnoff, & Andrews, 1991) and RHODES (Mirchandani & Head, 2001) are two other adaptive traffic controlling systems that use distributed control. These systems are run locally and coordination between intersections are done by communication between neighbors. As an example, when an intersection releases number of vehicles informs the next intersection about the time and number of vehicles to expect.

The use of artificial intelligence methods to control traffic signals started in 1990’s (Malej & Brodnik, 2007). Multiple optimization and estimation methods have been applied for adaptive control. Machine learning techniques are beneficial to create adaptive controllers with the ability to address unpredictable traffic condition issues. More explanation about applying machine learning techniques in controlling traffic signals will be presented in Section 2.

In this paper a review of intelligent methods have been applied for controlling traffic signals are presented. To have a comparison between three key intelligent methods have used in this area, Q-learning, neural network (NN), and fuzzy logic systems (FLS) controllers are implemented for a single intersection. Implementing these methods for an isolated intersection gives the opportunity to focus on the behavior of each method accurately. The reminder of this paper is organized as follows. Section 2 reviews the background machine learning theory and related work in applying machine learning techniques for traffic signal timing. In Section 3, Q-learning, NN, and FLS are used for timing an isolated traffic signal. Related results of experiments are presented in Section 4. Finally, Section 5 concludes the paper.

Section snippets

Background and related work

In this section, the related work and the background useful to understand the material presented in the rest of this paper will be covered.

Traffic signal controllers for an isolated intersection

QLC, NNC, FLC, and fixed-time controller are four controllers designed and implemented in this study for controlling signal times for an isolated intersection with four approaching links Fig. 4.

In most of the previous reviewed studies, the timing of signal is done by extending or terminating the current traffic signal phase. In this case it is not possible to have a estimation of the end of a phase at the start of that phase, therefore traffic signals with timer are not suitable for them.

Experiments results and discussion

For testing different controllers, an intersection with four approaching links and four phases (A, B, C, D) is considered. The cycle times are divided between aforementioned four phases. Zones are the areas that vehicles are released into the intersection, as shown in Fig. 4. Here, four different entrances to the intersection create four zones. The simulation is modeled in PARAMICS version 6.8. and all controllers are implemented in Matlab R2011b.

Two different scenarios are considered for

Conclusion and future work

The main purpose of this paper is to present a review of the previous works that applied computational intelligence methods for traffic signal controlling. There are many different intelligent techniques applied for traffic signal timing, but there is not any study to compare the performance of these techniques in a similar platform. In this paper, we present some of the most valuable previous works applied computational intelligence methods; specially, Q-learning, NN, and FLS, to traffic

References (88)

  • S.M. Rahman et al.

    Review of the fuzzy logic based approach in traffic signal control: Prospects in Saudi Arabia

    Journal of Transportation Systems Engineering and Information Technology

    (2009)
  • J.C. Spall et al.

    Traffic-responsive signal timing for system-wide traffic control

    Transportation Research Part C: Emerging Technologies

    (1997)
  • H. Yin et al.

    Urban traffic flow prediction using a fuzzy-neural approach

    Transportation Research Part C: Emerging Technologies

    (2002)
  • L.A. Zadeh

    The concept of a linguistic variable and its application to approximate reasoning: Part 1

    Information Sciences

    (1975)
  • LA. Zadeh

    The concept of a linguistic variable and its application to approximate reasoning: Part 2

    Information Sciences

    (1975)
  • LA. Zadeh

    The concept of a linguistic variable and its application to approximate reasoning: Part 3

    Information Science

    (1975)
  • J. Abdi et al.

    Short-term traffic flow forecasting: Parametric and nonparametric approaches via emotional temporal difference learning

    Neural Computing and Applications

    (2013)
  • Abdoos, M., Mozayani, N., & Bazzan, A. (2011). Traffic light control in non-stationary environments based on multi...
  • B. Abdulhai et al.

    Reinforcement learning for true adaptive traffic signal control

    Journal of Transportation Engineering

    (2003)
  • I. Arel et al.

    Reinforcement learning-based multi-agent system for network traffic signal control

    IET Intelligent Transport Systems

    (2010)
  • Balaji (2011). Distributed multi-agent based traffic management system (Ph.D. thesis). Singapore: University of...
  • P.G. Balaji et al.

    Distributed multi-agent type-2 fuzzy architecture for urban traffic signal control

  • P.G. Balaji et al.

    Multi-agent system in urban traffic signal control

    IEEE Computational Intelligence Magazine

    (2010)
  • C.M. Bishop

    Neural networks for pattern recognition

    (1995)
  • Cai, C. (2010). Adaptive traffic signal control using approximate dynamic programming (Ph.D. thesis). London:...
  • K. Capek et al.

    Evaluating the impacts of its applications using microscopic traffic simulators

    Advances in Transportation Studies

    (2011)
  • K.-H. Chao et al.

    An intelligent traffic light control based on extension neural network

  • Chin, D., Spall, J., & Smith, R. (1999). Evaluation of system-wide traffic signal control using stochastic optimization...
  • Y.-C. Chiou et al.

    Stepwise genetic fuzzy logic signal control under mixed traffic conditions

    Journal of Advanced Transportation

    (2013)
  • Chiu, S., & Chand, S. (1993). Self-organizing traffic control via fuzzy logic. In Proceedings of the 32nd IEEE...
  • M.C. Choy et al.

    Cooperative, hybrid agent architecture for real-time traffic signal control

    IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans

    (2003)
  • S.-B. Cools et al.

    Self-organizing traffic lights: A realistic simulation

  • Dai, Y., Zhao, D., & Yi, J. (2010). A comparative study of urban traffic signal control with reinforcement learning and...
  • A. Dekkers et al.

    Global optimization and simulated annealing

    Mathematical Programming

    (1991)
  • S. El-Tantawy et al.

    Multiagent reinforcement learning for integrated network of adaptive traffic signal controllers (MARLIN-ATSC): Methodology and large-scale application on downtown Toronto

    IEEE Transactions on Intelligent Transportation Systems

    (2013)
  • Favilla, J., Machion, A., & Gomide, F. (1993). Fuzzy traffic control: Adaptive strategies. In Second IEEE international...
  • N.H. Gartner et al.

    Evaluation of optimized policies for adaptive control strategy

    Transportation Research Record

    (1991)
  • M. Granberg et al.

    Traffic performance-based air quality management

    Advances in Transport

    (2001)
  • M. Granberg et al.

    Combined application of traffic microsimulation and street canyon dispersion models, and evaluation of the modelling system against measured data

    Advances in Transport

    (2000)
  • J. Heltimo et al.

    A microlevel method for road traffic noise prediction

    Advances in Transport

    (2002)
  • D. Houli et al.

    Multiobjective reinforcement learning for traffic signal control using vehicular ad hoc network

    EURASIP Journal on Advances in Signal Processing

    (2010)
  • Hu, Y., Thomas, P., & Stonier, R. (2007). Traffic signal control using fuzzy logic and evolutionary algorithms. In IEEE...
  • P. Hunt et al.

    The SCOOT on-line traffic signal optimisation technique (glasgow)

    Traffic Engineering & Control

    (1982)
  • V. Kononen et al.

    New methods for traffic signal control – Development of fuzzy controller

    (2000)
  • Cited by (105)

    • Extensions to traffic control modeling store-and-forward

      2023, Expert Systems with Applications
    • Type-2 fuzzy logic based transit priority strategy

      2022, Expert Systems with Applications
      Citation Excerpt :

      The authors used a genetic algorithm to solve the proposed model. There are also models, based on fuzzy logic, devoted to the signalized intersection control and bus priority problem (Chiou, Wang, and Lan (2005), Zaid and Al Othman (2011), Wang (2014), Araghi, Khosravi, and Creighton (2015)). To the best of our knowledge, the approach proposed in this paper represents the first attempt to solve the public transit priority problem by using Type-2 Fuzzy Logic, as a key system tool for giving bus priority.

    View all citing articles on Scopus
    View full text