Skip to main content
SpringerLink
Log in

Robustness through prior knowledge: using explanation-based learning to distinguish handwritten Chinese characters

  • Original Article
  • Published:
International Journal of Document Analysis and Recognition (IJDAR) Aims and scope Submit manuscript

Abstract

Handwritten Chinese character recognition is difficult due to the unstructured and noisy nature of its training examples. There are often too few training examples for a statistical learner like SVM to overcome the noise and extract useful information reliably. Existing prior domain knowledge represents a valuable source of information for classifying handwritten characters. Explanation-based learning (EBL) provides a way to incorporating prior domain knowledge into the learner. The dynamic bias formed by the interaction of domain knowledge with training examples can yield solution knowledge of potential higher quality. Two EBL approaches, one that uses a special feature kernel function in SVM, the other uses a conventional kernel for the SVM but provides additional preference in choosing the classification hyperplane, are reported.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bartlett, P., Shawe-Taylor, J.: Generalization performance of support vector machines and other pattern classifiers. In: Schölkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Advances in Kernel Methods: Support Vector Learning. MIT Press, Cambridge (1998)

  2. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001). http://www.csie.ntu.edu.tw/~cjlin/libsvm

  3. Cristianini, N., Shawe-Taylor, J., Elisseeff, A.: On kernel-target alignment. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems (NIPS), vol. 14, pp. 367–373. MIT Press, Cambridge (2002)

  4. DeCoste D. and Schölkopf B. (2002). Training invariant support vector machines. Mach. Learn. 46: 161–190

    Article  MATH  Google Scholar 

  5. Forsyth D.A. and Ponce J. (2002). Computer Vision—A Modern Approach. Prentice-Hall, Englewood Cliffs

    Google Scholar 

  6. Fung G.M., Mangasarian O.L. and Shavlik J.W. (2003). Knowledge-based nonlinear kernel classifiers. In: Schölkopf, B. and Warmuth, M.K. (eds) Learning Theory and Kernel Machines: 16th Annual Conference on Computational Learning Theory (COLT), Lecture Notes in Computer Science, vol. 2777, pp 102–113. Springer, Heidelberg

    Google Scholar 

  7. Fung G.M., Mangasarian O.L. and Shavlik J.W. (2003). Knowledge-based support vector machine classiers. In: Becker, S.T.S. and Obermayer, K. (eds) Advances in Neural Information Processing Systems (NIPS), vol. 15, pp 521–528. MIT Press, Cambridge

    Google Scholar 

  8. Kandola, J., Shawe-Taylor, J., Cristianini, N.: Optimizing kernel alignment over combinations of kernels. Technical Report NC-TR-02-121, NeuroCOLT (2002)

  9. LeCun Y., Bottou L., Bengio Y. and Haffner P. (1998). Gradient-based learning applied to document recognition. In: Proceedings of the IEEE 86(11): 2278–2324

    Article  Google Scholar 

  10. LeCun, Y., Jackel, L., Bottou, L., Brunot, A., Cortes, C., Denker, J., Drucker, H., Guyon, I., Muller, U., Sackinger, E., Simard, P., Vapnik, V.: Comparison of learning algorithms for handwritten digit recognition. In: Fogelman, F., Gallinari, P. (eds.) International Conference on Artificial Neural Networks, pp. 53–60 (1995)

  11. Lim, S.H., Wang, L.L., DeJong, G.: Explanation-based feature construction. In: International Joint Conference on Artificial Intelligence (IJCAI), pp. 931–936 (2007)

  12. Mangasarian O.L., Shavlik J.W. and Wild E.W. (2004). Knowledge-based kernel approximation. J. Mach. Learn. Res. 5: 1127–1141

    MathSciNet  Google Scholar 

  13. Mitchell T. (1997). Machine Learning. McGraw-Hill, New York

    MATH  Google Scholar 

  14. Ong, C.S., Smola, A.: Machine learning with hyperkernels. In: Proceedings of the Twentieth International Conference on Machine Learning (ICML) (2003)

  15. Saito T., Yamada H. and Yamamoto K. (1985). On the data base ETL 9 of handprinted characters in JIS Chinese characters and its analysis. IEICE Trans. J68-D(4): 757–764

    Google Scholar 

  16. Schölkopf, B., Burges, C., Vapnik, V.: Incorporating invariances in support vector learning machines. In: von der Malsburg, C., von Seelen, W., Vorbrüggen, J.C., Sendhoff, B. (eds.) Artificial Neural Networks—ICANN ’96, vol. 1112, pp. 47–52 (1996)

  17. Schölkopf B., Simard P., Vapnik V. and Smola A.J. (1998). Prior knowledge in support vector kernels. In: Jordan, M.I., Kearns, M.J., and Solla, S.A. (eds) Advances in Neural Information Processing Systems (NIPS), vol. 10, pp 640–646. MIT Press, Cambridge

    Google Scholar 

  18. Shawe-Taylor J., Bartlett P.L., Williamson R.C. and Anthony M. (1998). Structural risk minimization over data-dependent hierarchies. IEEE Trans. Inform. Theor. 44(5): 1926–1940

    Article  MATH  MathSciNet  Google Scholar 

  19. Shawe-Taylor J. and Cristianini N. (2002). On the generalisation of soft margin algorithms. IEEE Trans. Inform. Theor. 48(10): 2721–2735

    Article  MATH  MathSciNet  Google Scholar 

  20. Shi D., Damper R.I. and Gunn S.R. (2003). Off-line handwritten chinese character recognition by radical decomposition. ACM Trans. Asian Lang. Process. 2(1): 27–48

    Article  Google Scholar 

  21. Simard P.Y., Le Cun Y.A., Denker J.S. and Victorri B. (1998). Transformation invariance in pattern recognition—tangent distance and tangent propagation. In: Orr, G.B. and Müller, K.R. (eds) Neural Networks: Tricks of the Trade, Chap. 12., pp 239–274. Springer, Heidelberg

    Chapter  Google Scholar 

  22. Simard, P.Y., Steinkraus, D., Platt, J.: Best practice for convolutional neural networks applied to visual document analysis. In: International Conference on Document Analysis and Recogntion (ICDAR), pp. 958–962. IEEE Comput. Soc., Los Alamitos (2003)

  23. Sun, Q., DeJong, G.: Explanation-augmented SVM: an approach to incorporating domain knowledge into SVM learning. In: de Raedt, L., Wrobel, S. (eds) Proceedings of the 22nd International Machine Learning Conference. ACM Press, New York (2005)

  24. Teow, L.N., Loe, K.F.: Handwritten digit recognition with a novel vision model that extracts linearly separable features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 76–81 (2000)

  25. Vapnik V.N. (1998). Statistical Learning Theory. Wiley, New York

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Illinois at Urbana-Champaign, 201 North Goodwin Avenue, Urbana, IL, 61801, USA

    Qiang Sun, Li-Lun Wang, Shiau Hong Lim & Gerald DeJong

Authors
  1. Qiang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li-Lun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiau Hong Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gerald DeJong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li-Lun Wang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sun, Q., Wang, LL., Lim, S.H. et al. Robustness through prior knowledge: using explanation-based learning to distinguish handwritten Chinese characters. IJDAR 10, 175–186 (2007). https://doi.org/10.1007/s10032-007-0053-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10032-007-0053-1

Keywords

Search

Navigation