Skip to main content
SpringerLink
Log in

Support Vector–Quantile Regression Random Forest Hybrid for Regression Problems

  • Conference paper
Book cover Multi-disciplinary Trends in Artificial Intelligence (MIWAI 2014)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 8875))

Abstract

In this paper we propose a novel support vector based soft computing technique which can be applied to solve regression problems. Proposed hybrid outperforms previously known techniques in literature in terms of accuracy of prediction and time taken for training. We also present a comparative study of quantile regression, differential evolution trained wavelet neural networks (DEWNN) and quantile regression random forest ensemble models in prediction in regression problems. Intervals of the parameter values of random forest for which the performance figures of the Quantile Regression Random Forest (QRFF) are statistically stable are also identified. The effectiveness of the QRFF over Quantile Regression and DWENN is evaluated on Auto MPG dataset, Body fat dataset, Boston Housing dataset, Forest Fires dataset, Pollution dataset, by using 10-fold cross validation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Koenker, R.: Quantile Regression. Cambridge University Press (2005)

    Google Scholar 

  2. Breiman, L., Friedman, J., Stone, C.J., Olshen, R.A.: Classification and Regression Trees. Chapman & Hall/CRC (1984)

    Google Scholar 

  3. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann (1993)

    Google Scholar 

  4. Meinhausen, N.: Quantile Regression Forest. Journal of Machine Learning Research, 983–999 (2006)

    Google Scholar 

  5. Koenker, R., Basset, G.: Regression Quantiles. Econometrica: Journal of the Econometric Society Econometrica, 33–50 (1978)

    Google Scholar 

  6. Bhat, H.S., Kumar, N., Vaz, G.: Quantile Decision Trees (2011), http://faculty.ucmerced.edu/hbhat/BhatKumarVaz2011.pdf

  7. Ivakhnenko, A.G.: The group method of data handling - A rival of the method of stochastic approximation. Soviet Automatic Control 13(3), 43–55 (1966)

    Google Scholar 

  8. Breiman, L.: Random forests. Machine Learning 45, 5–32 (2001)

    Article  MATH  Google Scholar 

  9. Schulze, N.: Applied Quantile Regression: Microeconometric, Financial, and Environmental Analyses. PhD Dissertation, Faculty of Economics, Eberhard Karls University of Tübingen (2004)

    Google Scholar 

  10. Chauhan, N., Ravi, V., Karthikchandra, D.: Differential evolution trained wavelet neural networks: Application to bankruptcy prediction in banks. Expert Systems with Application 36, 7659–7665 (2009)

    Article  Google Scholar 

  11. Reddy, K.N., Ravi, V.: Kernel Group Method of Data Handling: Application to Regression Problems. In: Panigrahi, B.K., Das, S., Suganthan, P.N., Nanda, P.K. (eds.) SEMCCO 2012. LNCS, vol. 7677, pp. 74–81. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  12. Naveen, N., Ravi, V., Rao, C.R.: Rule extraction from DEWNN to Solve Classification and Regression Problems. In: Panigrahi, B.K., Das, S., Suganthan, P.N., Nanda, P.K. (eds.) SEMCCO 2012. LNCS, vol. 7677, pp. 206–214. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  13. Lin, Y., Jeon, Y.: Random Forest and adaptive nearest neighbors, Technical Report 1055, University of Wisconsin (2002)

    Google Scholar 

  14. Bache, K., Lichman, M.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine, CA, http://archive.ics.uci.edu/ml

  15. R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria (2013), http://www.R-project.org/

  16. Koenker, R.: quantreg: Quantile Regression. R package version 5.05 (2013), http://CRAN.R-project.org/package=quantreg

  17. Meinhausen, N.: quantregForest: Quantile Regression Forests. R package version 0.2-3 (2012), http://CRAN.R-project.org/package=quantregForest

  18. Mosteller, F., Tukey, J.W.: Data Analysis and Regression: A Second Course in Statistics. Addison-Wesley (1977)

    Google Scholar 

  19. Portnoy, S., Koenker, R.: The gaussian hare and the laplacian tortoise: Computability of squared-error versus absolute-error estimates. Statistical Science 12, 279–300 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  20. He, X., Ng, P., Portnoy, S.: Bivariate quantile smoothing splines. Journal of the Royal Statistical Society B 3, 537–550 (1998)

    Article  MathSciNet  Google Scholar 

  21. Biau, G.: Analysis of a Random Forests Model. Journal of Machine Learning Research 13, 1063–1095 (2012)

    MATH  MathSciNet  Google Scholar 

  22. Koenker, R., Ng, P., Portnoy, S.: Quantile smoothing splines. Biometrika 81, 673–680 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  23. Chaudhuri, P., Loh, W.: Nonparametric estimation of conditional quantiles using quantile regression trees. Bernoulli 8, 561–576 (2002)

    MATH  MathSciNet  Google Scholar 

  24. Barrodale, I., Roberts, F.: Solution of an overdetermined system of equations in the 1 norm. Communications of the ACM, 17, 319–320

    Google Scholar 

  25. Koenker, R., d’Orey, V.: Computing Regression Quantiles. Applied Statistics 36, 383–393 (1987)

    Article  Google Scholar 

  26. Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning (1995)

    Google Scholar 

  27. Srinivasan, D.: Energy demand prediction using GMDH networks. Neurocomputing 72(1-3), 625–629 (2008)

    Article  Google Scholar 

  28. Buchinsky, M.: Changes in U.S. Wage Structure 1963–1987: An Application of Quantile Regression. Econometrica 62(2), 405–458 (1994)

    Article  MATH  Google Scholar 

  29. Buchinsky, M.: The Dynamics of Changes in the Female Wage Distribution in the USA: A Quantile Regression Approach. Journal of Applied Econometrics 13(1), 1–30 (1997)

    Article  Google Scholar 

  30. Cortez, P., Morais, A.: A Data Mining Approach to Predict Forest Fires using Meteorological Data. In: Neves, J., Santos, M.F., Machado, J. (eds.) New Trends in Artificial Intelligence, Proceedings of the 13th EPIA 2007 - Portuguese Conference on Artificial Intelligence, Guimaraes, Portugal, pp. 512–523. APPIA (2007), http://www.dsi.uminho.pt/~pcortez/fires.pdf , ISBN-13 978-989-95618-0-9

  31. Arias, O., Hallock, K.F., Sosa-Escudero, W.: Individual Heterogeneity in the Returns to Schooling: Instrumental Variables Quantile Regression Using Twins Data. Empirical Economics 26(1), 7–40 (2001)

    Article  Google Scholar 

  32. Eide, E., Showalter, M.: The Effect of School Quality on Student Performance: A Quantile Regression Approach. Economics Letters 58(3), 345–350 (1998)

    Article  MATH  Google Scholar 

  33. Levin, J.: For Whom the Reductions Count: A Quantile Regression Analysis of Class Size on Scholastic Achievement. Empirical Economics 26(1), 221–246 (2001)

    Article  Google Scholar 

  34. Mueller, R.: Public- and Private-Sector Wage Differentials in Canada Revisited. Industrial Relations 39(3), 375–400 (2000)

    Google Scholar 

  35. Poterba, J., Rueben, K.: The Distribution of Public Sector Wage Premia: New Evidence Using Quantile Regression Methods, NBER Working Paper No. 4734 (1995)

    Google Scholar 

  36. Knight, K., Bassett, G., Tam, M.S.: Comparing Quantile Estimators for the Linear Model (2000) (preprint)

    Google Scholar 

  37. Newey, W., Powell, J.: Efficient Estimation of Linear and Type I Censored Regression Models Under Conditional Quantile Restrictions. Econometric Theory 6, 295–317 (1990)

    Article  MathSciNet  Google Scholar 

  38. Newey, W., Powell, J.: Asymmetric Least Squares Estimation and Testing. Econometrica 55, 819–847

    Google Scholar 

  39. Francke, T., López-Tarazón, J.A., Schröder, B.: Estimation of suspended sediment concentration and yield using linear models, random forests and quantile regression forests. Hydrological Processes

    Google Scholar 

  40. Archibald, S., Roy, D.P., Wilgen, V., Brian, W., Scholes, R.J.: What limits fire? An examination of drivers of burnt area in Southern Africa. Global Change Biology 15(3), 613–630 (2009)

    Article  Google Scholar 

  41. Satten, G.A., Datta, S., Moura, H., Woolfitt, A.R., Carvalho, M.G., Carlone, G.M., De, B.K., Pavlopoulos, A., Barr, J.R.: Standardization and denoising algorithms for mass spectra to classify whole-organism bacterial specimens. Bioinformatics 20(17), 3128–3136 (2004)

    Article  Google Scholar 

  42. Bossavy, A., Girard, R., Kariniotakis, G.: Forecasting uncertainty related to ramps of wind power production. In: European Wind Energy Conference and Exhibition, EWEC 2010, vol. 2 (2010)

    Google Scholar 

  43. Farquad, M.A.H., et al.: Support vector regression based hybrid rule extraction methods for forecasting. Expert Systems with Applications 37(8), 5577–5589 (2010)

    Article  Google Scholar 

  44. Reddy, K.N., Ravi, V.: Differential evolution trained kernel principal component WNN and kernel binary quantile regression: Application to banking. Knowledge-Based Systems 39, 45–56 (2012)

    Article  Google Scholar 

  45. Zhang, Z., Dai, G., Jordan, M.I.: Matrix-variate Dirichlet process mixture models. In: International Conference on Artificial Intelligence and Statistics, pp. 980–987 (2010)

    Google Scholar 

  46. Liaw, A., Wiener, M.: Classification and Regression by randomForest. R News 2(3), 18–22 (2002)

    Google Scholar 

  47. Benoit, D.F., Van den Poel, D.: Binary quantile regression: A Bayesian approach based on the asymmetric Laplace distribution. J. Appl. Econ. 27, 1174–1188 (2012)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center of Excellence in CRM and Analytics, Institute for Development and Research in Banking Technology, Castle Hill, Masab Tank, Hyderabad, 500057, A.P., India

    Ravi Vadlamani

  2. Department of Mathematics & Statistics, Indian Institute of Technology, Kanpur, 208016, U.P., India

    Anurag Sharma

Authors
  1. Ravi Vadlamani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anurag Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science and Automation, Indian Institute of Science, 560 012, Bangalore, India

    M. Narasimha Murty

  2. Faculty of Engineering and Information Technology, University of Technology Sydney, 2007, NSW, Australia

    Xiangjian He

  3. Department of Computer and Information Sciences, University of Hyderabad, 500046, Hyderabad, India

    Raghavendra Rao Chillarige

  4. LIP6, University Pierre et Marie Curie, 4 Place Jussieu, 75005, Paris, France

    Paul Weng

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Vadlamani, R., Sharma, A. (2014). Support Vector–Quantile Regression Random Forest Hybrid for Regression Problems. In: Murty, M.N., He, X., Chillarige, R.R., Weng, P. (eds) Multi-disciplinary Trends in Artificial Intelligence. MIWAI 2014. Lecture Notes in Computer Science(), vol 8875. Springer, Cham. https://doi.org/10.1007/978-3-319-13365-2_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-13365-2_14

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-13364-5

  • Online ISBN: 978-3-319-13365-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation