Smart data driven traffic sign detection method based on adaptive color threshold and shape symmetry
Introduction
Detection and recognition of road traffic sign constitute an important technological element in Advanced Driver Assistance Systems [1] (ADAS), which can provide real-time road sign information to vehicles and enhance driving safety [2]. Traffic signs are usually composed of specific shapes (round, square, and triangular) and colors (red, blue, and yellow), which have significant visual effect in the road environment. Therefore, traffic sign detection can be classified into color-based, shape-based and color-shape-based methods [3], [4].
In color-based traffic sign detection, the RGB images are usually transformed into other color space such as HSI [5], CIELab [6], and HSL [7]. Then the traffic signs are extracted by color thresholding segmentation based on smart data processing. Color-based traffic sign detection methods are easily affected by complex illumination conditions of traffic scenes. In shape-based traffic sign detection, geometric contour shapes of traffic signs are detected by means of template matching [8], geometric invariant moment [9], and geometric symmetry [6], [10]. Symmetry detection has better adaptability compared with template matching, geometric invariant moment in a complex illumination environment, but it requires higher computational complexity.
The existing color and shape based traffic sign detection methods have weaker adaptability under complex brightness conditions. In this paper, we propose an adaptive color threshold segmentation and high efficient shape symmetry algorithm to achieve robust traffic sign detection in a complex illumination environment. (1) In color thresholding segmentation: we calculate an adaptive threshold using the cumulative distribution function of image histogram; and we use the approximate maximum–minimum normalization method to process images according to the threshold. With this method, the interference of overexposure foreground objects and background is suppressed, so we can achieve better traffic sign segment effect in complex lighting environments. (2) In geometrical symmetry detection: by converting connected regions of a binary image into shape geometrical feature vectors, we use the statistical hypothesis testing method to judge the symmetry of connected components in a binary image, and extract symmetry components as the candidate ROIs of traffic signs. With this method, we realize higher efficient geometrical symmetry computation of the image than with the previous algorithm based on frequency domain.
The traffic sign detection process is shown in Fig. 1. The main contributions of this paper are included in the second and fifth steps (detailed algorithms are presented in Section 3).
Step 1: Traffic images are transformed into Red–Blue gray images by Red–Blue normalization processing [11];
Step 2: The adaptive segmentation threshold is calculated based on the cumulative distribution function of the histogram of Red–Blue images; and the images’ foreground enhancement is processed using the approximate maximum–minimum normalization method to suppress the images’ high brightness background;
Step 3: Candidate connected domains of traffic signs are extracted by morphological filtering and MSER [11];
Step 4: Enhanced processing of the connected domains is performed using convex operation;
Step 5: Symmetry statistical analysis and hypothesis testing are performed on the feature vectors of connected domains to detect shape symmetry and extract candidate ROIs of traffic signs.
Step 6: Traffic signs’ ROI is determined using geometric constraints.
The paper is organized as follows: Section 2 describes related works about traffic sign detection, Section 3 details the main contributions of this paper, Section 4 shows some experimental results of our method, and Section 5 presents the conclusions.
Section snippets
Related works
Traffic signs have significant color and shape features. As shown in Fig. 2, the GTSRB traffic sign dataset [12] includes prohibition signs (circular with a red outer ring), indication signs (circular with blue background) and warning signs (red upper triangle). Traffic sign detection methods are mainly classified into: color information based, shape information based, and fusion of color and shape information based detection algorithms.
Color information based traffic sign detection [13], [14],
Adaptive color threshold and shape symmetry testing in traffic sign detection
In this paper, we extract traffic signs’ regions (ROIs) through some prior fusion information of color and shape of traffic signs. This method has better adaptability to complex scenes such as illumination, rotation and scale change in road traffic environments.
Traffic sign dataset
German Traffic Sign Detection Benchmark (GTSDB) is a public traffic sign dataset. The dataset has 900 images with size of pixels (where 600 images are images for training and others are images for testing). The dataset contains various road scenes with dramatic changes in brightness and contrast conditions. The sizes of traffic signs’ candidate ROIs on the GTSDB range from 16 16 to 128 128. In the experiments, the traffic signs are manually marked on the images to facilitate
Conclusions
This paper presents a new traffic sign detection method based on adaptive color threshold segmentation and shape symmetry hypothesis testing by leveraging traffic signs and image data. First, in the color segmentation stage, the cumulative distribution function of the histogram is used to dynamically determine the segmentation threshold. And the approximate maximum–minimum normalization method is used to suppress the interference by high brightness area and background. So, the threshold segment
Acknowledgments
This work is supported by the Key Science-Technology Program of Zhejiang Province, China (2017C01022) and the National Natural Science Foundation of China (61370087).
Xianghua Xu is now a professor in the School of Computer Science at Hangzhou Dianzi University, China. He received his Bachelor degree from Hangzhou Institute of Electronic Engineering, and his Ph.D. in Computer Science from Zhejiang University, China. His current research interests include pattern recognition, parallel and distributed computing, and wireless sensor networks. He has published more than 100 peer reviewed journal and conference papers. He is a recipient of the Best Paper Award of
References (32)
- et al.
Traffic sign segmentation and classification using statistical learning methods
Neurocomputing
(2015) - et al.
A novel traffic sign detection method via color segmentation and robust shape matching
Neurocomputing
(2015) - Z. Malik, I. Siddiqi, Detection and recognition of traffic signs from road scene images, in: International Conference...
- L. Zhou, Z. Deng, LIDAR and vision-based real-time traffic sign detection and recognition algorithm for intelligent...
- et al.
Rapid multiclass traffic sign detection in high-resolution images
IEEE Trans. Intell. Transp. Syst.
(2014) - A. Mogelmose, D. Liu, M.M. Trivedi, Traffic sign detection for US roads: Remaining challenges and a case for tracking,...
- S. Sooksatra, T. Kondo, Red traffic light detection using fast radial symmetry transform, in: International Conference...
- M. Kehl, M. Enzweiler, B. Froehlich, et al. Vision-based road sign detection, in: IEEE International Conference on...
- et al.
Background on Traffic Sign Detection and Recognition
(2011) - X. Ma, C. Mu, Y. Wang, et al. Traffic sign detection and recognition using color standardization and Zernike moments,...
Real-Time detection and recognition of road traffic signs
IEEE Trans. Intell. Transp. Syst.
Road sign detection using eigen colour
Iet Comput. Vis.
Cited by (41)
MKL-SING: A data-driven approach of sign recognition for managing and improving public services
2023, Information Processing and ManagementAbessment of physical road infrastructure to support automated vehicles in an urban environment
2023, Transportation Research ProcediaGroup multi-scale attention pyramid network for traffic sign detection
2021, NeurocomputingDETECTION AND RECOGNITION OF TRAFFIC SIGNS FROM DATA COLLECTED BY THE MOBILE MAPPING SYSTEM
2024, International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS ArchivesEnhancing traffic sign recognition (TSR) by classifying deep learning models to promote road safety
2024, Signal, Image and Video Processing
Xianghua Xu is now a professor in the School of Computer Science at Hangzhou Dianzi University, China. He received his Bachelor degree from Hangzhou Institute of Electronic Engineering, and his Ph.D. in Computer Science from Zhejiang University, China. His current research interests include pattern recognition, parallel and distributed computing, and wireless sensor networks. He has published more than 100 peer reviewed journal and conference papers. He is a recipient of the Best Paper Award of IISWC’2012. He is the member of the IEEE, ACM.
Jiancheng Jin is now a graduate student in the School of Computer Science at Hangzhou Dianzi University, China. He received his Bachelor degree from Hangzhou Dianzi University. His current research interest is mainly focus on traffic sign detection in smart traffic systems.
Shanqing Zhang is now an associate professor in the School of Computer Science at Hangzhou Dianzi University, China. He received his Ph.D. in Computer Science from East China Normal University, China. His current research interests in pattern recognition. He has published more than 20 peer reviewed journal and conference papers.
Lingjun Zhang is now an assistant professor in the School of Computer Science at Hangzhou Dianzi University, China. He received his Ph.D. in Computer Science from Tianjing University, China. His current research interests in pattern recognition. He has published more than 10 peer reviewed journal and conference papers.
Shiliang Pu is the director of HIKVISION’s Research Institute. He received his Ph.D. in Applied Optics from the University of Rouen in France in 2005. His current research interests include image processing and pattern recognition. He is responsible for the company’s technology research and development work in video intelligent analysis, image processing, coding and decoding.
Zongmao Cheng is now an associate professor in the School of Science at Hangzhou Dianzi University, China. He received his Ph.D. in Statistics from East China Normal University, China. His current research interests in pattern recognition and data mining. He has published more than 20 peer reviewed journal and conference papers.