Abstract
This paper has two major parts. The first is an extensive analysis of the problem of induction, and the second part is a detailed study of selective induction. Throughout the paper we integrate a number of notions, mainly from artificial intelligence, but also from pattern recognition and cognitive psychology. The result is a synthetic view which exploits uncertainty, task-guidance, and biases such as language restriction. Some of the main themes and contributions are as follows. (1) Practical induction is really a problem of efficacy and efficiency (power). (2) Search in a space of hypothetical concepts is governed by acredibility function which combines various knowledge sources in a single subjective probability or belief measure μ. (3) The amount of knowledge supplied by various sources can often be quantified; these sources include various biases and the learning system itself. (4) Induction is equivalent to discovery of autility function u, which captures the purpose or goal of induction. (5) The difficulty of induction may be characterized by the form of u. Smooth or coherent functions mean selective induction, which has had the most attention in machine learning. (6) Systems for selective induction are more similar than commonly understood. By juxtaposing them we can discover similarities and improvements. (7) Our analysis suggests a number of incipient principles for powerful induction.
Article PDF
Similar content being viewed by others
References
Anderberg, M.R. (1973).Cluster analysis for applications. New York: Academic Press.
Angluin, D., & Smith, C.H. (1983). Inductive inference: Theory and methods.ACM Computing Surveys 15, 237–269.
Banerji, R.B. (1985). The logic of learning: A basis for pattern recognition and for improvement of performance.Advances in Computers,24, 177–216.
Bruner, J.S., Goodnow, J.J., & Austin, G.A. (1956).A study of thinking. New York: Wiley.
Cheeseman, P. (1985). In defense of probability.Proceedings of the Ninth International Joint Conference on Artificial Intelligence, (pp. 1003–1009). Los Angeles: Morgan-Kaufmann.
Christensen, R. (1964).Foundations of inductive reasoning. Berkeley: Entropy, Ltd.
Dietterich, T.G., London, B., Clarkson, K., & Dromey, G. (1982). Learning and inductive inference. In P.R. Cohen & E.A. Feigenbaum (Ed.),The handbook of artificial intelligence. Los Altos: Kaufmann.
Duda, R.O., & Hart, P.E. (1973).Pattern classification and scene analysis. New York: Wiley.
Easterlin, J.D., & Langley, P. (1985). A framework for concept formation.Proceedings of the Seventh Annual Conference of the Cognitive Science Society, (pp. 267–271). Irvine, CA.
Erman, L.D., Hayes-Roth, F., Lesser, V.R., & Reddy, D.R. (1980). The HEARSAY-II speech understanding system: Integrating knowledge to resolve uncertainty.Computing Surveys,12, 213–253.
Glass, A.L., & Holyoak, K.J. (1975). Alternative conceptions of semantic memory.Cognition,3, 313–339.
Holland, J.H. (1975).Adaptation in natural and artificial systems. Ann Arbor: University of Michigan Press.
Holte, R.C. (1986). Alternative information structures in incremental learning systems.Proceedings of the European Working Session on Learning.
Hunt, E.B., Marin, J., & Stone, P.J. (1966).Experiments in induction. New York: Academic Press.
Kanal, L., & Chandrasekaran, B. (1972). On linguistic, statistical and mixed models for pattern recognition. In S. Watanabe (Ed.),Frontiers of pattern recognition (pp. 163–192), New York: Academic Press.
Koestler, A. (1964).The act of creation: A study of the conscious and unconscious in science and art. New York: Macmillan (hard cover). (Some other editions omit the important Book Two).
Langley, P. (1985). Learning search strategies through discrimination.International Journal of Man-Machine Studies,18, 513–541.
Lenat, D.B. (1983). The role of heuristics in learning by discovery: Three case studies. In R.S. Michalski, J.G. Carbonell, & T.M. Mitchell (Eds.),Machine learning: An artificial intelligence approach (pp. 243–306). Palo Alto: Tioga.
Matheus, C.J. (1985). Views on concept representation and implications for machine learning. Unpublished manuscript.
Medin, D.L., & Smith, E.E. (1984). Concepts and concept formation.Annual Review of Psychology,35, 113–138.
Medin, D.L., & Wattenmaker, W.D. (1985). Category cohesiveness, theories, and cognitive archeology. In U. Neisser (Ed.),Concepts reconsidered: The ecological and intellectual bases of categories.
Michalski, R.S. (1983). A theory and methodology of inductive learning.Artificial Intelligence,20(2), 111–161. Reprinted in R.S. Michalski, J.G. Carbonell, & T.M. Mitchell (Eds.),Machine learning: An artificial intelligence approach (pp. 83–134). Palo Alto: Tioga.
Michalski, R.S., & Chilausky, R.L. (1980). Learning by being told and learning from examples: An experimental comparison of the two methods of knowledge acquisition in the context of developing an expert system for soybean disease diagnosis.International Journal Policy Analysis and Information Systems,4, 125–161.
Michalski, R.S., & Stepp, R.E. (1983). Learning from observation: Conceptual clustering, In R.S. Michalski, J.G. Carbonell, & T.M. Mitchell (Eds.),Machine learning: An artificial intelligence approach (pp. 331–363). Palo Alto: Tioga.
Michie, D. (1977). A theory of advice. In E.W. Elcock, & D. Michie (Eds.),Machine intelligence, 8 (pp. 151–168). American Elsevier.
Mitchell, T.M. (1978).Version spaces: An approach to concept learning. Unpublished doctoral dissertation, Stanford University, Stanford, California.
Mitchell, T.M. (1980).The need for biases in learning generalizations, (Technical Report No. CBMTR-117). New Brunswick: Rutgers University, Dept. of Computer Science.
Mitchell, T.M. (1982). Generalization as search.Artificial Intelligence,21, 203–226.
Murphy, G.L., & Medin, D.L. (1985). The role of theories in conceptual coherence.Psychological Review, 92.
O'Rorke, P. (1982).A comparative study of inductive learning systems AQ11P and ID-3 using a chess endgame test problem (Technical Report No. UIUCDCS-F-82–899 and ISG 82–2). Urbana-Champaign: University of Illinois, Dept. of Computer Science.
Quinlan, J.R. (1983). Learning efficient classification procedures and their application to chess end games. In R.S. Michalski, J.G. Carbonell, & T.M. Mitchell (Eds.),Machine learning: An artificial intelligence approach (pp. 463–482). Palo Alto: Tioga.
Quinlan, J.R. (1986). Induction of decision trees.Machine Learning 1, 81–106.
Rendell, L.A. (1977).A locally optimal solution of the fifteen puzzle produced by an automatic evaluation function generator (Technical Report No. CS-77–36). Waterloo, Ontario, Canada: University of Waterloo, Dept of Computer Science.
Rendell, L.A. (1981).An adaptive plan for state-space problems. Unpublished doctoral dissertation (Technical Report No. CS-81-13), University of Waterloo, Dept. of Computer Science, Waterloo, Ontario, Canada.
Rendell, L.A. (1983a). A new basis for state-space learning systems and a successful implementation,Artificial Intelligence,20, 369–392.
Rendell, L.A. (1983b).Conceptual knowledge acquisition in search (Technical Report No. CIS-83–15). Guelph, Ontario, Canada: University of Guelph, Dept. of Computing and Information Science. Also L. Bolc (Ed.).Computational models of learning, Springer-Verlag).
Rendell, L.A. (1985a). Genetic plans and the probabilistic learning system: Synthesis and results,Proceedings International Conference on Genetic Algorithms and their Applications (pp. 60–73).
Rendell, L.A. (1985b). Substantial constructive induction using layered information compression: Tractable feature formation in search,Proceedings of the Ninth International Joint Conference on Artificial Intelligence (pp. 650–658).
Rendell, L.A. (1986). Uncertain and incremental concept learning: Efficiency through model and data synergy. Unpublished manuscript.
Ritchie, G.D., & Hanna, F.K. (1984). AM: A case study in AI methodology.Artificial Intelligence,23, 249–268.
Samuel, A.L. (1967). Some studies in machine learning using the game of checkers II — recent progress.IBM Journal of Research & Development,11, 601–617.
Tou, T.T., & Gonzalez, R.C. (1974).Pattern recognition principles, Reading: Addison-Wesley.
Utgoff, P.E., & Mitchell, T.M. (1982). Acquisition of appropriate bias for inductive concept learning.Proceedings of the National Conference on Artificial Intelligence, 414–417.
Watanabe, S. (1969).Knowing and guessing: A formal and quantitative study, New York: Wiley.
Watanabe, S. (1972). Pattern recognition as information compression. In Watanabe, S. (Ed.).Frontiers of pattern recognition (pp. 561–567). New York: Academic Press.
Winston, P.H. (1984).Artificial intelligence (2nd ed.). Reading: Addison Wesley.
Wise, B.P., & Henrion, M. (1985). A framework for comparing uncertain inference systems to probability.Proceedings of Uncertainty and Probability in Artificial Intelligence Workshop (pp. 99–108).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Rendell, L. A general framework for induction and a study of selective induction. Mach Learn 1, 177–226 (1986). https://doi.org/10.1007/BF00114117
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00114117