Skip to main content
SpringerLink
Log in

Evolutionary optimization of multi-parametric kernel \(\epsilon\)-SVMr for forecasting problems

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

In this paper, we propose a novel multi-parametric kernel Support Vector Regression algorithm (SVMr) optimized with an evolutionary technique, specially well suited for forecasting problems. The multi-parametric SVMr model and the evolutionary algorithm proposed are both described in detail in the paper. In addition, several new bounds for the multi-parametric kernel considered are obtained, in such a way that the SVMr hyper-parameters’ search space is reduced. We present experimental evidences of the good performance of the evolutionary algorithm for optimizing the multi-parametric kernel, when compared to a standard SVMr with a Grid Search approach. Specifically, results in different real regression problems from public repositories are obtained, and also a real application focused on the short-term temperature prediction at Barcelona’s airport. The results obtained have shown the good performance of the multi-parametric kernel approach both in accuracy and computation time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Abe S (2005) Training of support vector machines with Mahalanobis kernels. Lect Notes Comput Sci 3697:571–576

    Google Scholar 

  • Akay MF (2009) Support vector machines combined with feature selection for breast cancer diagnosis. Expert Syst Appl 36(2):3240–3247

    Article  Google Scholar 

  • Asuncion A, Newman DJ (2007) UCI Machine Learning Repository 2007.University of California, School of Information and Computer Science, Irvine, CA. http://www.ics.uci.edu/~mlearn/MLRepository.html

  • Brodic D (2010) Optimization of the anisotropic Gaussian kernel for text segmentation and parameter extraction. Theor Comput Sci IFIP Adv Inf Commun Technol 323:140–152

    Article  Google Scholar 

  • Chang CC, Lin C-J (2011) LIBSVM : a library for support vector machines. ACM Trans Intell Syst Technol 2:1–27

    Article  Google Scholar 

  • Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46:131–159

    Article  MATH  Google Scholar 

  • Eiben AE, Smith JE (2003) Introduction to evolutionary computing. Springer, Berlin

    MATH  Google Scholar 

  • Fauvel M, Villa A, Chanussot J, Benediktsson JA (2010) Mahalanobis kernel for the classification of hyperspectral images. In: Proceedings of the IEEE international geoscience and remote sensing symposium, Honolulu, Hawaii, pp 3724–3727

  • Friedrichs F, Igel C (2005) Evolutionary tuning of multiple SVM parameters. Neurocomputing 64:107–117

    Article  Google Scholar 

  • Gascón-Moreno J, Ortiz-García EG, Salcedo-Sanz S, Paniagua-Tineo A, Saavedra-Moreno B, Portilla-Figueras JA (2011) Multi-parametric Gaussian kernel function optimization for \(\epsilon\)-SVMr using a genetic algorithm, IWANN 2011, Lecture Notes in Computer Science, vol 6692, pp 113–120

  • Gijsberts A, Metta G, Rothkrantz L (2010) Evolutionary optimization of least-squares support vector machines. Ann Inf Syst 8(Special Issue on Data Mining):277–297

    Article  Google Scholar 

  • He W, Wang Z, Jiang H (2008) Model optimizing and feature selecting for support vector regression in time series forecasting. Neurocomputing 72(1–3):600–611

    Article  Google Scholar 

  • Hou S, Li Y (2009) Short-term fault prediction based on support vector machines with parameter optimization by evolution strategy. Expert Syst Appl 36(10):12383–12391

    Article  Google Scholar 

  • Ortiz-Garcia EG, Salcedo-Sanz S, Pérez-Bellido AM, Portilla-Figueras JA (2009) Improving the training time of support vector regression algorithms through novel hyper-parameters search space reductions. Neurocomputing 72(1–3):3683–3691

    Article  Google Scholar 

  • Pai PF, Hong WC, Lee YS (2005) Determining parameters of support vector machines by genetic algorithms—applications to reliability prediction. Int J Oper Res 2(1):1–7

    MATH  Google Scholar 

  • Phienthrakul T, Kijsirikul B (2005) Evolutionary strategies for multi-scale radial basis function kernels in support vector machines. In: Proceedings of the 2005 conference on genetic and evolutionary computation, GECCO ’05

  • Phienthrakul T, Kijsirikul B (2010) Evolutionary strategies for hyperparameters of support vector machines based on multi-scale radial basis function kernels. Soft Comput 14(7):681–699

    Article  Google Scholar 

  • Rieger C, Zwicknagl B (2009) Deterministic error analysis of support vector regression and related regularized kernel methods. J Mach Learn Res 10:2115–2132

    MathSciNet  MATH  Google Scholar 

  • Salcedo-Sanz S, Ortiz-García G Emilio, Pérez-Bellido AM, Portilla-Figueras A, Prieto L (2011) Short term wind speed prediction based on evolutionary support vector regression algorithms. Expert Syst Appl 38(4):4052–4057

    Article  Google Scholar 

  • Shamsheyeva A, Sowmya A (2004) The anisotropic Gaussian kernel for SVM classification of HRCT images of the lung. In: Proceedings of the intelligent sensors, sensor networks and information processing conference

  • Shao X, Cherkassky V (1999) Multi-resolution support vector machine. In: Proceedings of international joint conference on neural networks 2, pp 1065–1070

  • Shitong W, Jiagang Z, Chung FL, Qing L, Dewen H (2005) Theoretically optimal parameter choices for support vector regression machines with noisy input. Soft Comput 9(10):732–741

    Article  MATH  Google Scholar 

  • Smola AJ, Schölkopf B (1998) A tutorial on support vector regression. Stat Comput

  • Smola AJ, Murata N, Scholkopf B, Muller K (1998) Asymptotically optimal choice of \(\epsilon\)-loss for support vector machines. In: Proceedings of the 8th international conference on artificial neural networks, perspectives in neural computing

  • StatLib DataSets Archive (2012). http://lib.stat.cmu.edu/datasets

  • Villa A, Fauvel M, Chanussot J, Gamba P, Benediktsson JA (2008) Gradient Optimization for multiple kernel’s parameters in support vector machines classification. In: IEEE international geoscience and remote sensing symposium, pp 224–227

  • Wang S, Zhu J, Chung FL, Dewen Hu (2006) Experimental study on parameter choices in norm-r support vector regression machines with noisy input. Soft Comput 10(3):219–223

    Article  Google Scholar 

  • Wu CL, Chau KW, Li YS (2008) River stage prediction based on a distributed support vector regression. J Hydrol 358(1–2):96–111

    Article  Google Scholar 

  • Wu GH, Tzeng GH, Lin RH (2009) A novel hybrid genetic algorithm for kernel function and parameter optimization in support vector regression. Expert Syst Appl 36(3):4725–4735

    Article  Google Scholar 

  • Yang Z, Guo J, Xu W, Nie X, Wang J, Lei J (2006) Multi-scale support vector machine for regression estimation. In: Lecture Notes in Computer Science, vol 3971, pp 1030–1037

  • Zhao YP, Sun JG (2011) Multikernel semiparametric linear programming support vector regression. Expert Syst Appl 38(3):1611–1618

    Article  Google Scholar 

  • Zheng D, Wang J, Zhao Y (2006) Non-flat function estimation with a multi-scale support vector regression. Neurocomputing 70:420–429

    Article  Google Scholar 

  • Zhou L, Lai KK, Yu L (2009) Credit scoring using support vector machines with direct search for parameters selection. Soft Comput 13(2):149–155

    Article  MATH  Google Scholar 

Download references

Acknowledgments

This work has been partially supported by Spanish Ministry of Science and Innovation, under project number ECO2010-22065-C03-02.

Author information

Authors and Affiliations

  1. Department of Signal Processing and Communications, Escuela Politécnica Superior, Universidad de Alcalá, 28871, Alcalá de Henares, Madrid, Spain

    J. Gascón-Moreno, S. Salcedo-Sanz, L. Carro-Calvo, B. Saavedra-Moreno & A. Portilla-Figueras

  2. Department of Signal Processing and Communications, Universidad Carlos III de Madrid, Leganés, Madrid, Spain

    E. G. Ortiz-García

Authors
  1. J. Gascón-Moreno
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. E. G. Ortiz-García
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. Salcedo-Sanz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. L. Carro-Calvo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. B. Saavedra-Moreno
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. A. Portilla-Figueras
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. Gascón-Moreno.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gascón-Moreno, J., Ortiz-García, E.G., Salcedo-Sanz, S. et al. Evolutionary optimization of multi-parametric kernel \(\epsilon\)-SVMr for forecasting problems. Soft Comput 17, 213–221 (2013). https://doi.org/10.1007/s00500-012-0886-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-012-0886-5

Keywords

Search

Navigation