Abstract
In this paper we present a primal-dual decomposition algorithm for support vector machine training. As with existing methods that use very small working sets (such as Sequential Minimal Optimization (SMO), Successive Over-Relaxation (SOR) or the Kernel Adatron (KA)), our method scales well, is straightforward to implement, and does not require an external QP solver. Unlike SMO, SOR and KA, the method is applicable to a large number of SVM formulations regardless of the number of equality constraints involved. The effectiveness of our algorithm is demonstrated on a more difficult SVM variant in this respect, namely semi-parametric support vector regression.
References
Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Belmont (1999)
Chang, C.-C., Lin, C.-J.: Training ν-support vector classifiers: Theory and algorithms. Neural Computation 13(9), 2119–2147 (2001)
Chang, C.-C., Lin, C.-J.: LIBSVM – a library for support vector machines, version 2.71 (August 2004), http://www.csie.ntu.edu.tw/~cjlin/libsvm/
Chen, P.-H., Lin, C.-J., Schölkopf, B.: A tutorial on ν-support vector machines. Applied Stochastic Models in Business and Industry (2005) (to appear)
Collobert, R., Bengio, S.: SVMTorch: Support vector machines for large-scale regression problems. Journal of Machine Learning Research 1, 143–160 (2001)
Dong, J., Krzyzak, A., Suen, C.Y.: Fast svm training algorithm with decomposition on very large data sets. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(4), 603–618 (2005)
Friess, T., Cristianini, N., Campbell, C.: The kernel adatron algorithm: a fast and simple learning procedure for support vector machine. In: International Conference on Machine Learning, pp. 188–196 (1998)
Joachims, T.: Making large-scale SVM learning practical. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods, pp. 42–56. MIT Press, Cambridge (1999)
Mangasarian, O.L., Musicant, D.R.: Successive overrelaxation for support vector machines. IEEE Transactions on Neural Networks 10, 1032–1037 (1999)
Osuna, E., Freund, R., Girosi, F.: Support vector machines: Training and applications. Technical Report AIM-1602 (1997)
Platt, J.: Fast training of support vector machines using sequential minimal optimization. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods, pp. 185–208. MIT Press, Cambridge (1998)
Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)
Smola, A.J., Friess, T., Schölkopf, B.: Semiparametric support vector and linear programming machines. In: Advances in Neural Information Processing Systems 11, pp. 585–591 (1998)
Vanderbei, R.J.: LOQO: An interior point code for quadratic programming. Technical Report SOR 94-15, Princeton University, NJ (1994)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kienzle, W., Schölkopf, B. (2005). Training Support Vector Machines with Multiple Equality Constraints. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds) Machine Learning: ECML 2005. ECML 2005. Lecture Notes in Computer Science(), vol 3720. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11564096_21
Download citation
DOI: https://doi.org/10.1007/11564096_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29243-2
Online ISBN: 978-3-540-31692-3
eBook Packages: Computer ScienceComputer Science (R0)