Accurate, data-efficient, unconstrained text recognition with convolutional neural networks

Highlights

• We propose a novel neural network architecture for unconstrained text recognition.

• Our proposed architecture uses only the highly efficient convolutional primitives

• We achieve state-of-the-art results on seven public benchmark datasets.

• Our proposed model has won the ICFHR2018 Competition on Automated Text Recognition.

Abstract

Unconstrained text recognition is an important computer vision task, featuring a wide variety of different sub-tasks, each with its own set of challenges. One of the biggest promises of deep neural networks has been the convergence and automation of feature extractors from input raw signals, allowing for the highest possible performance with minimum required domain knowledge. To this end, we propose a data-efficient, end-to-end neural network model for generic, unconstrained text recognition. In our proposed architecture we strive for simplicity and efficiency without sacrificing recognition accuracy. Our proposed architecture is a fully convolutional network without any recurrent connections trained with the CTC loss function. Thus it operates on arbitrary input sizes and produces strings of arbitrary length in a very efficient and parallelizable manner. We show the generality and superiority of our proposed text recognition architecture by achieving state-of-the-art results on seven public benchmark datasets, covering a wide spectrum of text recognition tasks, namely: Handwriting Recognition, CAPTCHA recognition, OCR, License Plate Recognition, and Scene Text Recognition. Our proposed architecture has won the ICFHR2018 Competition on Automated Text Recognition on a READ Dataset.

Introduction

Text recognition is considered one of the earliest computer vision tasks to be tackled by researches. For more than a century now since its inception as a field of research, researchers never stopped working on it. This can be contributed to two important factors.

First, the pervasiveness of text and its importance to our everyday life, as a visual encoding of language that is used extensively in communicating and preserving all kinds of human thought. Second, the necessity of text to humans and its pervasiveness have led to big adequacy requirements over its delivery and reception which has led to the large variability and ever-increasing visual forms of text. Text can originate either as printed or handwritten, with large possible variability in the handwriting styles, the printing fonts, and the formatting options. Text can be found organized in documents as lines, tables, forms, or cluttered in natural scenes. Text can suffer countless types and degrees of degradations, viewing distortions, occlusions, backgrounds, spacings, slantings, and curvatures. Text from spoken languages alone (as an important subset of human-produced text) is available in dozens of scripts that correspond to thousands of languages. All this has contributed to the long-standing complicated nature of unconstrained text recognition.

Since a deep Convolutional Neural Network (CNN) won the ImageNet image classification challenge [1], Deep Learning based techniques have invaded most tasks related to computer vision, either equating or surpassing all previous methods, at a fraction of the required domain knowledge and field expertise. Text recognition was no exception, and methods based on CNNs and Recurrent Neural Networks (RNNs) have dominated all text recognition tasks like OCR [2], Handwriting recognition [3], scene text recognition [4], and license plate recognition [5], and have became the de facto standard for the task.

Despite their success, one can spot a number of shortcomings in these works. First, for many tasks, an RNN is required for achieving state-of-the-art results which brings-in non-trivial latencies due to the sequential processing nature of RNNs. This is surprising, given the fact that, for pure-visual text recognition long range dependencies have no effect and only local neighborhood should affect the final frame or character classification results. Second, for each of these tasks, we have a separate model that can, with its own set of tweaks and tricks, achieve state-of-the-art result in a single task or a small number of tasks. Thus, No single model is demonstrated effective on the wide spectrum of text recognition tasks. Choosing a different model or feature-extractor for different input data even inside a relatively limited problem like text recognition is a great burden for practitioners and clearly contradicts the idea of automatic or data-driven representation learning promised by deep learning methods.

In this work, we propose a novel, purely feed forward neural network architecture for efficient, generic, unconstrained text recognition. Our proposed architecture is a fully convolutional CNN [6] that consists mostly of depthwise separable convolutions with novel inter-layer residual connections [7] and gating, trained on full line or word labels using the Connectionist Temporal Classification (CTC) loss [8]. We also propose a set of generic data augmentation techniques that are suitable for any text recognition task and show how they affect the performance of the system. We demonstrate the superior performance of our proposed system through extensive experimentation on seven public benchmark datasets. We were also able, for the first time, to demonstrate human-level performance on the reCAPTCHA dataset proposed recently in a Science paper [9], which is more than 20% absolute increase in CAPTCHA recognition rate compared to their proposed RCN system. We also achieve state-of-the-art performance in SVHN [10] (the full sequence version), the unconstrained settings of IAM English offline handwriting dataset [11], KHATT Arabic offline handwriting dataset [12], University of Washington (UW3) OCR dataset [13], AOLP license plate recognition dataset [14] (in all divisions). Our proposed system has also won the ICFHR2018 Competition on Automated Text Recognition on a READ Dataset [15] achieving more than 25% relative decrease in Character Error Rate (CER) compared to the entry achieving the second place.

To summarize, we address the unconstrained text recognition problem. In particular, we make the following contributions:

• We propose a novel neural network architecture that is able to achieve state-of-the-art performance, with feed forward connections only (no recurrent connections), and using only the highly efficient convolutional primitives.

• We propose a set of data augmentation procedures that are generic to any text recognition task and can boost the performance of any neural network architecture on text recognition.

• We conduct an extensive set of experiments on seven benchmark datasets to demonstrate the validity of our claims about the generality of our proposed architecture. We also perform an extensive ablation study on our proposed model to demonstrate the importance of each of its submodules.

Section 2 gives an overview of related work on the field. Section 3 describes our architecture design and its training process in details. Section 4 describes our extensive set of experiments and presents its results.