Skip to main content
Log in

Boosting the Hardware-Efficiency of Cascade Support Vector Machines for Embedded Classification Applications

  • Published:
International Journal of Parallel Programming Aims and scope Submit manuscript

Abstract

Support Vector Machines (SVMs) are considered as a state-of-the-art classification algorithm capable of high accuracy rates for a different range of applications. When arranged in a cascade structure, SVMs can efficiently handle problems where the majority of data belongs to one of the two classes, such as image object classification, and hence can provide speedups over monolithic (single) SVM classifiers. However, the SVM classification process is still computationally demanding due to the number of support vectors. Consequently, in this paper we propose a hardware architecture optimized for cascaded SVM processing to boost performance and hardware efficiency, along with a hardware reduction method in order to reduce the overheads from the implementation of additional stages in the cascade, leading to significant resource and power savings. The architecture was evaluated for the application of object detection on \(800\times 600\) resolution images on a Spartan 6 Industrial Video Processing FPGA platform achieving over 30 frames-per-second. Moreover, by utilizing the proposed hardware reduction method we were able to reduce the utilization of FPGA custom-logic resources by \(\sim \)30%, and simultaneously observed \(\sim \)20% peak power reduction compared to a baseline implementation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Cortes, C., Vapnik, V.: Support-vector networks. J. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  2. Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Discov. 2, 121–167 (1998)

    Article  Google Scholar 

  3. Osuna, E., Freund, R., Firosi, F.: Training support vector machines: an application to face detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 130–136 (1997)

  4. Wu, J., Rehg, J.M.: Chapter 8 object detection. In: Ensemble Machine Learning: Methods and Applications, 1st ed., pp. 225–250. Springer-Verlag, New York (2012)

    Chapter  Google Scholar 

  5. Heisele, B., Serre, T., Prentice, S., Poggio, T.: Hierarchical classification and feature reduction for fast face detection with support vector machines. Pattern Recognit. 36(9), 2007–2017 (2003)

    Article  Google Scholar 

  6. Kukenys, I., McCane, B.: Classifier cascades for support vector machines. In: International Conference on Image and Vision Computing, pp. 1–6 (2008)

  7. Ma, Y., Ding, X.: Face Detection Based on Cost-Sensitive Support Vector Machines. In: First International Workshop on Pattern Recognition with Support Vector Machines, pp. 260–267 (2002)

    Chapter  Google Scholar 

  8. Cadambi, S., et al.: A Massively Parallel FPGA-Based Coprocessor for Support Vector Machines. In: IEEE International Symposium on Field Programmable Custom Computing Machines (FCCM), pp. 115–122 (2009)

  9. Pina-Ramirez, O., Valdes-Cristerna, R., Yanez-Suarez, O.: An FPGA implementation of linear kernel support vector machines. In: IEEE International Conference on Reconfigurable Computing and FPGA’s, pp. 1–6 (2006)

  10. Ruiz-Llata, M., Guarnizo, G., Yébenes-Calvino, M.: FPGA implementation of a support vector machine for classification and regression. In: International Conference on Neural Networks, pp. 1–5 (2010)

  11. Fowers, J., Brown G., Cooke, P., Stitt, G.: A performance and energy comparison of FPGAs, GPUs, and multicores for sliding-window applications. In: ACM/SIGDA International Symposium on Field Programmable Gate Arrays (FPGA ’12), pp. 47–56 (2012)

  12. Cooke, P., Fowers, J., Brown, G., Stitt G.: A Tradeoff Analysis of FPGAs, GPUs, and Multicores for Sliding-Window Applications. In: ACM Transactions on Reconfigurable Technology Systems, vol. 8, no. 1 (2015)

    Article  Google Scholar 

  13. Kyrkou, C., Bouganis, C.-S., Theocharides, T.: An Embedded Hardware-Efficient Architecture for Real-Time Cascade Support Vector Machine Classification. In: International Conference on Embedded Computer Systems (SAMOS), pp. 129–136 (2013)

  14. Burges, C.J.C.: Simplified support vector decision rules. In: International Conference on Machine Learning, pp. 71–77 (1996)

  15. Sahbi, H., Geman, D., Boujemaa, N.: Face detection using coarse-to-fine support vector classifiers. In: International Conference on Image Processing, pp. 925–928 (2001)

  16. Romdhani, S., Torr, P., Schölkopf, B., Blake, A.: Efficient face detection by a cascaded support-vector machine expansion. R. Soc. Lond. Proc. Ser. A 460(2051), 3283–3297 (2004)

    Article  MathSciNet  Google Scholar 

  17. Ojala, T., Pietikainen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recognit. 26(1), 51–59 (1996)

    Article  Google Scholar 

  18. Hadid, A., Pietikainen, M., Ahonen, T.: A Discriminative Feature Space for Detecting and Recognizing Faces. In: IEEE Conference on Computer Vision and Pattern Recognition (2004)

  19. Pietikainen, M., Abdenour, H., Zhao, G., Ahonen, T.: Computer Vision Using Local Binary Patterns, vol. 40. Springer-Verlag, London (2011)

    Google Scholar 

  20. Catanzaro, B., Sundaram, N., Keutzer, K.: Fast support vector machine training and classification on graphics processors. In: International conference on Machine Learning, pp. 104–111 (2002)

  21. Zhao, H.-X., Magoules, F.: Parallel Support Vector Machines on Multi-core and Multiprocessor Systems. In: International Conference on Artificial Intelligence and Applications (2010)

  22. Anguita, D., Boni, A., Ridella, S.: A digital architecture for support vector machines: theory, algorithm, and FPGA implementation. IEEE Trans. Neural Netw. 14(5), 993–1009 (2003)

    Article  Google Scholar 

  23. Genov, R., Gauwengerghs, G.: Kerneltron: support vector machines in silicon. IEEE Trans. Neural Netw. 14, 1426–1434 (2003)

    Article  Google Scholar 

  24. Mahmoodi, D., Soleimani, A., Khosravi, H., Taghizadeh, M.: FPGA simulation of linear and nonlinear support vector machine. J. Softw. Eng. Appl. 4, 320–328 (2011)

    Article  Google Scholar 

  25. Biasi, I., Boni, A., Zorat, A.: A reconfigurable parallel architecture for SVM classification. In: IEEE International Joint Conference on Neural Networks, pp. 2867–2872 (2005)

  26. Groleat, T., Arzel, M., Vaton, S.: Harware Acceleration of SVM-based traffic classification on FPGA. In: International Wireless Communications and Mobile Computing Conference, pp. 443–449 (2012)

  27. Ruiz-Llata, M., Yebenes-Calvino, M.: FPGA Implementation of Support Vector Machines for 3D Object Identification. In: International Conference on Artificial Neural Networks (2009)

  28. Reyna-Rojas, R., Houzet, D., Dragomirescu, D., Carlier, D., Ouadjaout, S.: Object recognition system-on-chip using the support vector machines. EURASIP J. Adv. Signal Process. 2005, 993–1004 (2005)

    Article  Google Scholar 

  29. Graf, H.P., et al.: A Massively Parallel Digital Learning Processor. In: Annual Conference on Neural Information Processing Systems (NIPS), pp. 529–536 (2008)

  30. Boni, A., Pianegiani, F., Petri, D.: Low-power and low-cost implementation of SVMs for smart sensors. IEEE Trans. Instrum. Meas. 56(1), 39–44 (2007)

    Article  Google Scholar 

  31. Anguita, D., Ghio, A., Pischiutta, S., Ridella, S.: A Hardware-friendly Support Vector Machine for Embedded Automotive Applications. In: International Joint Conference on Neural Networks (2007)

  32. Meher, P.K., Valls, J., Tso-Bing, J., Sridharan, K., Maharatna, K.: 50 Years of CORDIC: Algorithms, Architectures, and Applications. In: IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 56, no. 9, pp. 1893–1907 (2009)

    Article  MathSciNet  Google Scholar 

  33. Khan, F., Arnold, M., Pottenger, W.: Finite Precision Analysis of Support Vector Machine Classification in Logarithmic Number Systems. In: Euromicro Symposium on Digital System Design (2004)

  34. Khan, F., Arnold, M., Pottenger, W.: Hardware-based support vector machine classification in logarithmic number systems. In: IEEE International Symposium on Circuirs and Systems (2005)

  35. Boni, A., Zorat, A.: FPGA Implementation of Support Vector Machines with Pseudo-Logarithmic Number Representation. In: International Joint Conference on Neural Networks (2006)

  36. Anguita, D., Ghio, A., Pischiutta, S.: A learning machine for resource-limited adaptive hardware. In: Second NASA/ESA Conference on Adaptive Hardware and Systems (2007)

  37. Ghio, A., Pischiutta, S.: A Support Vector Machine based pedestrian recognition system on resource-limited hardware architectures. In: Research in Microelectronics and Electronics Conference (2007)

  38. Anguita, D., Pischiutta, S., Ridella, S., Sterpi, D.: Feed-forward support vector machine without multipliers. IEEE Trans. Neural Netw. 17, 1328 (2006)

    Article  Google Scholar 

  39. Carpenter, A.: CUSVM: A CUDA Implementation of Support Vector Machines (2009)

  40. Bauer, S., Kohler, S., Doll, K., Brunsmann, U.: FPGA-GPU Architecture for Kernel SVM Pedestrian Detection. In: Computer Vision and Pattern Recognition Workshops (2010)

  41. Papadonikolakis, M., Bouganis, C.-S.: Novel cascade FPGA accelerator for support vector machines classification. Trans. Neural Netw. Learn. Syst. 23(7), 1040–1052 (2012)

    Article  Google Scholar 

  42. Maghazeh, A., Bordoloi, U., Eles, P., Peng, Z.: General Purpose Computing on Low-Power Embedded GPUs : Has It Come of Age? In: International Conference on Embedded Computer Systems (SAMOS XIII) (2013)

  43. Kyrkou, C., Theocharides, T.: A parallel hardware architecture for real-time object detection with support vector machines. IEEE Trans. Comput. 61(6), 831–842 (2012)

    Article  MathSciNet  Google Scholar 

  44. Kryjak, T., Komorkiewicz, M., Gorgon, M.: FPGA implementation of real-time head-shoulder detection using local binary patterns, SVM and foreground object detection. In: International Conference on Design and Architectures for Signal and Image Processing (2012)

  45. Microblaze Soft Processor. Xilinx, San Jose, CA. http://www.xilinx.com/tools/microblaze.htm

  46. CBCL Face Database #1, MIT Center for Biological and Computation Learning. http://cbcl.mit.edu/software-datasets/FaceData2.html

  47. CBCL PEDESTRIAN DATABASE #1, MIT Center for Biological and Computation Learning. http://iris.usc.edu/Vision-Users/OldUsers/bowu/DatasetWebpage/dataset.html

  48. Bao Face Database. http://www.facedetection.com/downloads/BaoDataBase.zip

  49. CMU and MIT Face Database. http://vasc.ri.cmu.edu/idb/html/face/frontal_images/

  50. PETS 2012 Database. In: 14th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance. http://www.pets2012.net/

Download references

Acknowledgements

This work was supported in part by the ERC Advanced Grant “Fault-Adaptive”, ERC grant agreement no 291508.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Christos Kyrkou.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kyrkou, C., Theocharides, T., Bouganis, CS. et al. Boosting the Hardware-Efficiency of Cascade Support Vector Machines for Embedded Classification Applications. Int J Parallel Prog 46, 1220–1246 (2018). https://doi.org/10.1007/s10766-017-0514-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10766-017-0514-1

Keywords

Navigation