1 Introduction

In recent years, barcode technology has been widely used in our daily life. Indeed, barcode detection is one of today’s most important technologies. By using computer image processing technology, such as image enhancement, segmentation, and recognition, we can achieve a fully automatic procedure for barcode detection. Many barcode detection algorithms have been proposed in previous research. However, these algorithms have advantages and disadvantages.

A projection method is proposed to extract the barcode region in [1]. First, the barcode image edge is extracted by the edge detector. Second, the processed image is projected in the horizontal and vertical direction. This algorithm is based on the assumption that the barcode edge is a complicating factor and is less dense than other areas of the barcode and that other areas have relatively smooth gray edge pixels. This method can achieve good performance in a simple background image. However, such performance is difficult to achieve in a real application with a complex background. Reference [2] proposed a new algorithm. The image is first divided into 32 × 32 patches and then the angle of the parallel lines in each patch is determined and combined with the angle close to the block to get the bar code area. However, the algorithm cannot ensure the segmentation of various small pieces that contain only part of the barcode, which leads to angle miscalculation and incomplete extraction of the barcode from some images. Reference [3] proposed a fast barcode location method based on the discrete cosine transform (DCT) domain. First, the DCT is used to distinguish the barcode area and the other regions. Then the barcode angle and position are detected in the original image using the extracted region features. However, this algorithm is not robust because the weight matrix coefficients are not adaptive. Telkin and Coughlan [4] proposed a new algorithm, which is divided into two steps. First, the noise of the image is reduced by Gauss smoothing, the gradient amplitude of the image is calculated, and the gradient amplitude forms the edge enhanced image. The second step is to determine the barcode region by calculating the four directions of the pixel with the gradient value higher than a certain threshold. However, this algorithm is not sufficient for different kinds of barcodes that exist at the same time. Reference [5] proposed a method based on edge detection and machine learning. The Hough transform and neural network are obtained to determine the region of the barcode line. This algorithm is adequate for detection of the general image, but it has a poor effect on an image with dense text, and the result of the detection depends on the results of the Hough transform.

In this paper, we propose a method for barcode detection of camera-captured images, which includes Hough transform, local binary pattern (LBP) and color feature extraction, and support vector machine (SVM) classification. In our method, the Hough transform is used for angle correction of barcode images, and the LBP and color feature are extracted for machine learning model training and the SVM classifier. After angle correction by the Hough transform, the original image with barcode blocks is detected by the classifier. Then the blocks are merged with the extracted adjacent block region, and the isolated region is excluded. The obtained results prove that our algorithm can effectively detect the barcode region in a complex background.

The remainder of the paper is organized as follows: The optimizing algorithm is addressed in Sect. 2. The detailed simulation and results analysis are shown in Sect. 3. The conclusion is provided in the last section.

2 Proposed Method

2.1 Barcode Detection

The frame of the proposed method for detecting barcodes is shown in Fig. 1. To obtain the SVM classifier, we extract the color feature and LBP feature from image blocks and then train the features to achieve the SVM model. In a complex background, the direction of an image has an impact on detecting the barcode. First, the angle of an input image is corrected by the two-dimension Hough transform. Second, the sub-block of the corrected image is verified with the mode of the trained SVM. In the end, the classification results of each block are scanned, all isolated blocks are removed, and the neighbored area is merged to achieve the region of the barcode. The positive and negative samples (14 × 14 pixels) shown in Figs. 2 and 3 serve to extract image features by LBP and thus obtain the trained mode of SVM.

Fig. 1.
figure 1

Frame of proposed method

Full size image
Fig. 2.
figure 2

Positive block image

Full size image
Fig. 3.
figure 3

Negative block image

Full size image

Local Binary Pattern Feature.

The features of the positive and negative example are extracted by the LBP method. The features include a 4 × 4 × 58 dimension vector that becomes the training database. The LBP feature is defined by the local texture, which has the characteristic of low computing complexity, and is not affected by the degree of image shading. The LBP has a good effect on texture feature extraction. Indeed, the LBP has many applications in the field of machine vision. The local texture feature of a gray image is described by a joint distribution of gray value of pixels within the region, as follows:

$$ T = t\left( {g_{c} ,g_{0} , \ldots ,g_{p - 1} } \right) $$
(1)

In Eq. 1, T represents the joint probability distribution, and g represents the gray value of the central pixel in the local area (p = 0, …, P − 1); the gray value of the P pixel points and the image pixels are evenly distributed on the circle with the radius of the center of R. The value of P and R determines the size and range of the local field distribution, as shown in Fig. 4.

Fig. 4.
figure 4

Circularly symmetric regions have differences (P, R)

Full size image

Reference [6] introduced the derived formula:

$$ {\text{LBP}}_{\text{P,R}} = \sum\nolimits_{\text{P = 0}}^{\text{P - 1}} {{\text{s}}(g_{p} - g_{c} )2^{P} } $$
(2)

Defined here:

$$ S(x) = \{_{0,\;x < 0}^{1,\;x \ge 0} $$
(3)

By Eq. 2, the two values of the local texture model described by the LBP operator are obtained by comparing the gray values of the field pixels with the central pixel. The LBP operator is defined in the circular location and can be better defined with the invariance of the texture feature operator, in which the pixel values in the field are obtained by linear interpolation. In describing the features, the LBP has better performance in the details.

Color Feature Extraction.

Since the majority of the barcode is composed of black and white lines, the color feature is extremely good for the identification of a barcode; the 256 dimension gray scale histogram method based on the statistical gray scale is used to extract the color feature of the block example. The statistics of the number of blocks in the same pixel, divided by the total number of block pixels, yields the percentage of different grays in the block, and then we obtain the 256 dimensional feature. The histogram of the horizontal coordinates is the gray t, and the vertical coordinate is the proportion of the gray scale, the gray value of the probability density of P(r), and the following relationship is established:

$$ P(r) = \begin{array}{*{20}c} { \lim } \\ {\varDelta r \to 0} \\ \end{array} \frac{A(r + \varDelta r) - A(r)}{\varDelta r} $$
(4)
$$ \int {}_{r\,\hbox{min} }^{r\,\hbox{max} } \,P(r)dr = 1 $$
(5)

In Eqs. 4 and 5, A is the area of the pixel block and r is gray value.

2.2 Support Vector Machine

The LBP feature and color feature set is trained by the SVM classifier, and then we get the model which can be used to predict the block image. The implementation process of the SVM classifier is described below.

SVM is developed by the optimal classification of linear separable conditions, and the basic idea of SVM can be illustrated by the two dimensional cases. In Fig. 5, the solid points and the hollow points are represented by the positive sample and negative sample, and then the line for classification; the other subscript number line of H represents a straight line with the nearest sample and parallel to the classification line. The distance between the two lines of H 1 and H 2 is called the gap width (margin). The significance of the optimal classification line is that the classification line can divide examples into two categories and let the margin be maximum.

Fig. 5.
figure 5

Optimal classification surface

Full size image

Reference [7] provides a detailed introduction. The above problems are summarized as quadratic function problems with inequality constraints. The classification function is shown in Eq. 6.

$$ f(x) = {\text{sgn}}\{ (w^{*} x) + b^{*} \} = {\text{sgn}}\{ \sum\limits_{i = 1}^{n} {a_{i}^{*} y_{i} (x_{i} x) + b^{*} } \} $$
(6)

In this formula, the summations are adequate for the support vector (nonsupport vector a i is zero). b * is the classification threshold, which can be obtained with any support vector. The basic idea of SVM can be described as follows: First, the input space is transformed into a high dimensional space by nonlinear transformation, and then the optimal classification surface is obtained in the new space. The optimal classification surface can be used to classify the sample.

2.3 Hough Transform

Image rectifications are based on two-dimensional Hough transform. In this section, we use the Canny edge detector algorithm to get the edge map with the implementation offered by OpenCV library [8]. Then we use the Hough transform to get the angle of the barcode line. We then rotate image B to 0° or 90°, which facilitates detecting the barcode.

Angle Detection Using Hough Transform.

The classical Hough transform [10] is a method for regular feature extraction; this method is used in image processing and computer vision to detect regular shapes. The Hough transform for line detection identifies a set of linear shapes L in a given image B by using a voting procedure method. The equation of a generic line 1 ∈ L in B can be defined as follows:

$$ \sigma = x*\cos \theta + y*\sin \theta $$
(7)

where \( \sigma \ge 0 \), the distance of l from the origin, is image B and \( \theta \in \;[0,2\pi ) \) is the angle of 1.

In this paper, we use the 2 dimensional Hough transform [5] to detect the angle of the barcode. Let the 2 dimensional Hough transform space H be the \( (\sigma ,\theta ) \) plane; for any point, \( (x_{1} ,y_{1} ) \) can be corresponded to a sinusoid in H. If two points \( (x_{1} ,y_{1} ) \), \( (x_{2} ,y_{2} ) \) belong to the identical line l, their homologous sinusoid will be defined in a point \( \sigma_{1} ,\theta_{1} \in H \). We define \( D_{H} \) as the accumulator matrix for the 2 dimensional Hough space \( H = (\sigma ,\theta ) \) in image B; every element \( b \in D_{H} \) in position \( (\sigma_{b} ,\theta_{b} ) \) denotes a potential line in B having same angle θ b and distance of axes \( \sigma_{b} \). The goal of the angle detection phase is to change the angle of image B, and then the barcode in image B can be detected easily.

Because the density characteristics of the Hough transform of the barcode constitute a special character, we adopted the intensity characteristic to distinguish the location, which is barcode location or not. As shown in Fig. 6, the barcode line has a difference \( \sigma \), so we let the element \( D_{H} \) be a regular grid of cells C, and every point \( D_{H} (\sigma ,\theta ) \) refers to the intensity of image B. Let \( C_{i} (x,y) \) be the value of the element of \( C_{i} \) in position \( (x,y) \) with \( 0 \le \;x < vx \) and 0 ≤ y < vy (every cell \( C_{i} \in C \)); the value is the sum of \( D_{H} (x,y) \), as \( \sum\limits_{{}} {\begin{array}{*{20}c} {x < vx,y < v{\text{y}}} \\ {x = 0,y = 0} \\ \end{array} } D_{H} (x,y) \). To get the angle of the barcode line in image B, we use the histogram statistics method (implemented by OpenCV) [8] to obtain the intensity Threshold T. Then we can get the map, which can describe the angle \( \theta \) of the barcode line.

Fig. 6.
figure 6

Angle detection using Hough transform

Full size image
$$ \theta = \theta_{{c_{i} (x,y)}} \quad \square \;if\;\quad C_{i} \left( {x,y} \right)\; > \;T $$
(8)

Figure 6 denotes how the regular grid of cells C is superimposed over the Hough accumulator matrix D H and how the elements are \( C_{i} \left( {x,y} \right) \) mapped into the (σ, θ) plane.

Rotating Images.

We determine the angle of the line of the barcode, then change the angle of image B to detect the barcode location. We just need to set the barcode angle, such as 0°, 90°, which is good for our detection. Our method of rotating the image is the linear spatial filtering of the OpenCV library [8]. Image B rotates to a 0° or 90° image, which will help us more easily detect the barcode. The resulting images of rotation are shown in Fig. 7.

Fig. 7.
figure 7

After detecting the angle of the barcode, the rotated image by the angle; the (a) row is the original image and the (b) row is the rotated image

Full size image

Our method to detect the angle of the image is almost exactly the same. The result of our method is shown in Fig. 8.

Fig. 8.
figure 8

The angle detection accuracy in the deviation of angle

Full size image

Figure 8 shows that our method results in exactly the same ±5° deviation of angle as the method of [5]. Our method does not need to use the neural network, so we can cut the time for training and detecting. Thus, our method can detect the angle more quickly.

2.4 Using the SVM Classifier to Judge Pieces of the Image

As shown in Fig. 9, after rotating image B, we divide image B into small blocks (every block is 14 × 14 pixels in this paper). Then we extract the LBP feature and color feature for each block and combine the feature data of the block. Using the SVM classifier (trained in Sect. 2.2), we judge each block of image B.

Fig. 9.
figure 9

Block image with a complex background

Full size image

In this experiment, the size of positive and negative samples is 14 × 14 pixels. The positive samples are about 1000 blocks and the negative samples are about 2000 blocks. The positive samples contain a barcode in the test image database. The negative samples contain other elements without a barcode. Every block uses Eq. (2) to detect the LBP feature (the argument for LBP feature extraction is P = 8, R = 1.0). The color feature for every block is extracted by Eq. (4). After extracting the feature, every block has a 4 × 4 × 58 dimension LBP feature vector and a 256 dimension color feature vector; we mix these features to achieve a 1184 dimension feature and enter the 1184 dimension feature in the SVM classifier; our method adopts C-SVM; the argument in SVM has the penalty factor C and RBF kernel function (coefficient k, k is the feature dimension).

In the results of the classification, ‘1’ represents the location of the barcode; other locations are represented by ‘0’. The classified result is shown in Fig. 10.

Fig. 10.
figure 10

Classified result by SVM

Full size image

2.5 Post-processing

The result distinguished by the SVM classifier is not 100 % complete; it is necessary to provide the corresponding post-processing. The post-processing steps are as follows:

  1. 1.

    Scan the classification results of each block in turn. Remove all isolated blocks of 1 and 0.

  2. 2.

    For every block, if more than half the results differ from the 4 neighborhood region, this should change the overall result.

  3. 3.

    Merge the unicom area and remove the small area unicom region.

  4. 4.

    The rectangle marked as the unicom area yields the location of the barcode.

3 Experiments and Results Analysis

3.1 Datasets

In this section, we provide the datasets used to evaluate the performance of our proposed model. In our experiments, we employ a standard 1D barcode dataset, the WWU Muenster Barcode Database [9], and the size of the image is 640 × 480 pixels. Since our method involves an SVM algorithm, we split the dataset image into training and test sets: We randomly select 50 % of the dataset’s images as the training set, then artificially extract the barcode as the positive example and extract another location as the negative example; the remaining 50 % is used as the test set. The following paragraph introduces the WWU Muenster Barcode Database [9]. WWU has 1055 1D barcode images, acquired using a Nokia N95 mobile phone. All images in the database contain a non-blurred EAN or UPC-A barcode.

3.2 Result

In this section, we compare the results of our proposed method with other methods. We compare the results of before rotating the image in Fig. 11. Results for conditions to detect barcodes are shown in Fig. 12.

Fig. 11.
figure 11

Comparison of images without rotation and our rotated result

Full size image
Fig. 12.
figure 12

Comparison of experimental results

Full size image

Figure 11 shows the result of detecting the barcode in the image with rotation. The (a) column is the original picture, the (b) column is ref. [5], the (c) column is our method result without rotation, and the (d) column is our result with rotation.

Figure 12 shows the result of detecting the barcode in a complex image. The first row is the big barcode image, the second row is the small barcode image, and the third row is the rotated image. The (a) column is the original picture, the (b) column is ref. [4], the (c) column is ref. [5], and the (d) column is our method result.

As shown in Table 1, we compare the algorithm performance in the WWU Muenster Barcode Database. Our algorithm can detect the barcode in complex conditions and reduce the impact of complex backgrounds on detection results.

Table 1. Comparison of the experimental results
Full size table

The detection accuracy of our method is 95.34 % in the WWU Muenster Barcode Database. With a complex background, our method can detect the barcode more accurately. With the method of refs. [4, 5], the performance depends on the results of the Hough transform. In the presence of a more text intensive area, the performance of those methods will be poor. In our method, we use the LBP feature to describe the image features, which can distinguish the interference region. Our method has better robustness and can detect barcodes more accurately than the method of refs. [4, 5], which is good for barcode recognition.

4 Conclusion

In this paper, we have presented a simple method to detect the region of the barcode and proved the effectiveness of the proposed approach by using the available WWU barcode dataset. The proposed method uses LBP image feature exaction and SVM classification to locate the regions of barcodes, and the angle invariable is obtained by the Hough transform. The experimental results show that our method is precisely able to detect the regions of multiple barcodes in an image with a complex background and the angle invariable of a barcode is also verified. The accuracy of detection using our approach is higher than for approaches used in earlier research.