Abstract
The naive Bayesian classifier provides a very simple yet surprisingly accurate technique for machine zijian@deakin.edu.au Some researchers have examined extensions to the naive Bayesian classifier that seek to further improve the accuracy. For example, a naive Bayesian tree approach generates a decision tree with one naive Bayesian classifier at each leaf. Another example is a constructive Bayesian classifier that eliminates attributes and constructs new attributes using Cartesian products of existing attributes. This paper proposes a simple, but effective approach for the same purpose. It generates a naive Bayesian classifier committee for a given classification task. Each member of the committee is a naive Bayesian classifier based on a subset of all the attributes available for the task. During the classification stage, the committee members vote to predict classes. Experiments across a wide variety of natural domains show that this method significantly increases the prediction accuracy of the naive Bayesian classifier on average. It performs better than the two approaches mentioned above in terms of higher prediction accuracy.
Chapter PDF
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Almuallim, H. and Dietterich, T.G.: Efficient algorithms for identifying relevant features.Proceedings of the gth Canadian Conference on Artificial Intelligence. Vancouver, BC: Morgan Kaufmann (1992) 38–45.
Breiman, L., Friedman, J.H., Olshen, R.A., and Stone, C.J.: Classification And Regression Trees. Belmont, CA: Wadsworth (1984).
Breiman, L.: Bagging predictors. Machine Learning. 24 (1996) 123–140.
Cestnik, B., Kononenko, I., and Bratko, I.: Assistant 86: A knowledge-elicitation tool for sophisticated users. In I. Bratko & N. Lavrač (Eds.), Progress in Machine Learning — Proceedings of the 2nd European Working Session on Learning (EWSL87). Wilmslow, UK: Sigma Press (1987) 31–45.
Chatfield, C.: Statistics for Technology: A Course in Applied Statistics. London: Chapman and Hall (1978).
Domingos, P. and Pazzani, M.: Beyond independence: Conditions for the optimality of the simple Bayesian classifier. Proceedings of the 13th International Conference on Machine Learning. San Francisco, CA: Morgan Kaufmann (1996) 105–112.
Duda, R.O. and Hart, P.E.: Pattern Classification and Scene Analysis. New York: John Wiley (1973).
Fayyad, U.M. and Irani, K.B.: Multi-interval discretization of continuous-valued attributes for classification learning. Proceedings of the 13th International Joint Conference on Artificial Intelligence. Morgan Kaufmann (1993) 1022–1027.
Freund, Y. and Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Unpublished manuscript, available from the authors' home pages (“http://www.research.att.com/≈yoav,≈schapire”) (1996a).
Freund, Y. and Schapire, R.E.: Experiments with a new boosting algorithm. Proceedings of the 13th International Conference on Machine Learning. Morgan Kaufmann (1996b) 148–156.
John, G.H., Kohavi, R., and Pfleger, K.: Irrelevant features and the subset selection problem. Proceedings of the 11th International Conference on Machine Learning. San Francisco, CA: Morgan Kaufmann (1994) 121–129.
Kira, K. and Rendell, L.A.: The feature selection problem: Traditional methods and a new algorithm. Proceedings of the 10th National Conference on Artificial Intelligence. Menlo Park, CA: AAAI Press/Cambridge, MA: MIT Press (1992) 129–134.
Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. Proceedings of the 14th International Joint Conference on Artificial Intelligence. San Mateo, CA: Morgan Kaufmann (1995) 1137–1143.
Kohavi, R.: Scaling up the accuracy of naive-Bayes classifiers: A decision-tree hybrid. Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining. Menlo Park, CA: The AAAI Press (1996) 202–207.
Kononenko, I.: Comparison of inductive and naive Bayesian learning approaches to automatic knowledge acquisition. In B. Wielinga et al. (Eds.), Current Trends in Knowledge Acquisition. Amsterdam: IOS Press (1990).
Kononenko, I.: Semi-naive Bayesian classifier. Proceedings of European Conference on Artificial Intelligence (1991) 206–219.
Langley, P., Iba, W.F., and Thompson, K.: An analysis of Bayesian classifiers. Proceedings of the 10th National Conference on Artificial Intelligence. Menlo Park, CA: The AAAI Press (1992) 223–228.
Langley, P.: Selection of relevant features in machine learning. Proceeding of the AAAI Fall Symposium on Relevance, New Orleans, LA: The AAAI Press (1994).
Langley, P. and Sage, S.: Induction of selective Bayesian classifiers. Proceedings of the 10th Conference on Uncertainty in Artificial Intelligence. Seattle, WA: Morgan Kaufmann (1994) 339–406.
Merz, C.J. and Murphy, P.M.: UCI Repository of Machine Learning Databases [http://www.ics.uci.edu/≈mleaxn/MLRepository.html]. Irvine, CA: University of California, Department of Information and Computer Science (1997).
Pazzani, M.J.: Constructive induction of Cartesian product attributes. Proceedings of the Conference, ISIS'96: Information, Statistics and Induction in Science. Singapore: World Scientific (1996) 66–77.
Quinlan, J.R.: Bagging, boosting, and C4.5. Proceedings of the 13th National Conference on Artificial Intelligence, Menlo Park: The AAAI Press (1996) 725–730.
Schapire, R.E., Freund, Y., Bartlett, P., and Lee W.S.: Boosting the margin: A new explanation for the effectiveness of voting methods. Proceedings of the 11th International Conference on Machine Learning. Morgan Kaufmann (1997) 322–330.
Ting, K.M.: Discretization of continuous-valued attributes and instance-based learning (Technical Report 491). Sydney, Australia: University of Sydney, Basser Department of Computer Science (1994).
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zheng, Z. (1998). Naive bayesian classifier committees. In: Nédellec, C., Rouveirol, C. (eds) Machine Learning: ECML-98. ECML 1998. Lecture Notes in Computer Science, vol 1398. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0026690
Download citation
DOI: https://doi.org/10.1007/BFb0026690
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-64417-0
Online ISBN: 978-3-540-69781-7
eBook Packages: Springer Book Archive