Abstract
Support Vector Machines (SVMs) have become a popular tool for learning with large amounts of high dimensional data. But it may sometimes be preferable to learn incrementally from previous SVM results, as SVMs which involve the solution of a quadratic programming problem suffer from the problem of large memory requirement and CPU time when trained in batch mode on large data sets. And the SVMs may be used in online learning setting. In this paper an approach for incremental learning with Support Vector Machines is presented. We define the normal solution of the incremental learning for SVMs which is defined as the solution minimizing a given positive-definite quadratic form in the coordinates of the difference vector between the normal vectors at the (k-1)-th and k-th incremental step and discuss the relation to standard SVM. It was shown that concept learned at last step will not change if new data satisfy separable condition and empirical evidence is given to prove that this approach can effectively deal with changes in the target concept that are results of the incremental learning setting according to three evaluation criteria: stability, improvement and recoverability.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)
Xiao, R., Wang, J., Zhang, F.: An approach to incremental SVM learning algorithm. In: ICTAI 2000, pp. 268–273. IEEE Computer Society, Canada (2000)
Mitra, P., Murthy, C.A., Sankar, K.: Data Condensation in Large Databases by Incremental Learning with Support Vector Machines. In: ICPR 2000, pp. 2708–2711. IEEE Computer Scociety, Barcelona (2000)
Syed, N.A., Liu, H., Sung, K.: Incremental learning with support vector machines. In: Proceedings of the Workshop on Support Vector Machines at the international Joint Conference on Artificial Intelligence (IJCAI 1999), Stockholm, Sweden (1999)
Cauwenberghs, G., Poggio, T.: Incremental and Decremental Support Vector Machine Learning. In: NIPS 2000, pp. 409–415. MIT Press, USA (2000)
Syed, N.A., Liu, H., Sung, K.: Handling concept drifts in incremental learning with support vector machines. In: Proceedings of 1st Intl. Conf. on Knowledge Discovery and Data Mining, AAAI, pp. 317–321 (1999)
Domeniconi, C., Gunopulos, D.: Incremental Support Vector Machine Construction. In: Cercone, N., Lin, T.Y., Wu, X. (eds.) Proceedings of the 2001 IEEE International Conference on Data Mining, San Jose, California, USA, pp. 589–592. IEEE Computer Society, Los Alamitos (2001)
Ralaivola, L., d’Alch-Buc, F.: Incremental Support Vector Machine Learning: A Local Approach. In: Dorffner, G., Bischof, H., Hornik, K. (eds.) ICANN 2001. LNCS, vol. 2130, pp. 322–330. Springer, Heidelberg (2001)
Osuna, E., Freund, R., Girosi, F.: An improved training algorithm for support vector machines. In: Proceedings, IEEE Workshop on Neural Networks for Signal Processing, pp. 276–285 (1997)
Platt, J.C.: Fast training of support vector machines using sequential minimum optimization. In: Advances in Kernel Methods–Support Vector Learning, pp. 185–208. MIT Press, Cambridge (1998)
Campbell, C., Cristianini, N., Smola, A.: Query learning with large margin classifiers. In: Proceddings, 17th International Conference on Machine Learning, pp. 111–118. Morgan Kaufmann, San Francisco (2000)
Martin, M.: On-line support vector machines for function approximation, Technical Report (2002), http://www.lsi.upc.es/dept/techreps/ps/R02-11.pdf
Fine, S., Scheinberg, K.: Incremental learning and selective sampling via parametric optimization framework for SVM. In: Advances in Neural Information Processing Systems 14, pp. 705–711. MIT Press, Cambridge (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, Y., Chen, Q., Tang, Y., He, Q. (2004). An Incremental Updating Method for Support Vector Machines. In: Yu, J.X., Lin, X., Lu, H., Zhang, Y. (eds) Advanced Web Technologies and Applications. APWeb 2004. Lecture Notes in Computer Science, vol 3007. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24655-8_45
Download citation
DOI: https://doi.org/10.1007/978-3-540-24655-8_45
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21371-0
Online ISBN: 978-3-540-24655-8
eBook Packages: Springer Book Archive