Abstract
One of the obstacles to widely using first-order logic languages is the fact that relational inference is intractable in the worst case. This paper presents an any-time relational inference algorithm: it proceeds by stochastically sampling the inference search space, after this space has been judiciously restricted using strongly-typed logic-like declarations.
We present a relational learner producing programs geared to stochastic inference, named STILL, to enforce the potentialities of this framework. STILL handles examples described as definite or constrained clauses, and uses sampling-based heuristics again to achieve any-time learning.
Controlling both the construction and the exploitation of logic programs yields robust relational reasoning, where deductive biases are compensated for by inductive biases, and vice versa.
Article PDF
Similar content being viewed by others
References
Anglano, C., Giordana, A., Lo Bello, G., & Saitta, L. (1998). An experimental evaluation of coevolutive concept learning. In J. Shavlik (Ed.), Proc. of the 15th ICML (pp. 19–27). Morgan Kaufmann.
Blockeel, H. & De Raedt, L. (1998). Top-down induction of first order logical decision trees. Artificial Intelligence, 101(1–2), 285–297.
Borgida, A. (1996). On the relative expressiveness of description logics and predicate logics. Artificial Intelligence, 82, 353–367.
Botta, M., Giordana, A., & Piola, R. (1997). FONN: Combining first order logic with connectionist learning. In D. Fisher (Ed.), Proc. of ICML-97 (pp. 48–56). Morgan Kaufmann.
Boufkhad, Y. (1998). Algorithms for propositional KB approximation. Proc. of AAAI-98 (pp. 280–285). AAAI Press.
Brachman, R. J. (1977). A structural paradigm for representing knowledge. Ph.D. Thesis, Harvard University, Cambridge, MA.
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and Regression by tree. Belmont California: Wadsworth.
Cadoli, M. (1993). Semantical and computational aspects of Horn approximations. Proc. of IJCAI-93 (pp. 39–44). Morgan Kaufmann.
Clocksin, W. F. & Mellish, C. S. (1981). Programming in Prolog. New-York: Springer-Verlag.
Crawford, J. M. & Etherington, D. W. (1998). A non-deterministic semantics for tractable inference. Proc. of AAAI-98 (pp. 286–291). AAAI Press.
Crawford, J. M. & Kuiper, B. (1989). Toward a theory of access-limited logic for knowledge representation. Proc. of KR-89 (pp. 67–78).
Cussens, J. (1997). Part-of-speech tagging using Progol. In N. Lavrac & Saso Dzeroski (Eds.), Proc. of ILP-97 (pp. 93–108). LNCS, Vol. 1297, Springer Verlag.
De Raedt, L. (1996). Induction in logic. In Proc. of 3rd International Workshop on Multistrategy Learning (pp. 29–38). AAAI Press.
De Raedt, L. & Džeroski, S. (1994). First order jk-clausal theories are PAC-learnable. Artificial Intelligence, 70, 375–392.
De Raedt, L., Idestam-Almquist, P., & Sablon, G. (1997). Θ-subsumption for structural matching. In M. Van Someren & G. Widmer (Eds.), Proc. of the 9th European Conference on Machine Learning (pp. 73–84). Springer Verlag.
Dietterich, T. G., Lathrop, R.,& Lozano-Perez, T. (1997). Solving the multiple-instance problem with axis-parallel rectangles. Artificial Intelligence, 89(1–2), 31–71.
Dolsak, D. & Muggleton, S. (1991). The application of ILP to finite element mesh design. In S. Muggleton (Ed.), Proc. of the 1st ILP (pp. 453–472). Springer Verlag.
Emde, W. & Wettscherek, D. (1996). Relational instance based learning. In L. Saitta (Ed.), Proc. of the 13th ICML (pp. 122–130). Morgan Kaufmann.
Esposito, F., Laterza, A., Malerba, D., & Semeraro, G. (1996). Locally finite, proper and complete operators for refining datalog programs. In Z. W. Raz & M. Michalewicz (Eds.), Foundations of intelligent systems (pp. 468–478). Springer Verlag.
Feigenbaum, E. (1977). The art of artificial intelligence: themes and case studies in knowledge engineering. Proc. of IJCAI'77 (pp. 1014–1029).
Giordana, A. & Neri, F. (1994) Search intensive concept induction. Evolutionary Computation, 3(4), 375–416.
Giordana, A., Botta, M., & Saitta, L. (1999). An experimental study of phase transitions in matching. In Proc. of IJCA'99 (pp. 1199–1203). Morgan Kaufmann.
Haussler, D. (1988). Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36, 177–221.
Jaffar, J. & Lassez, J. L. (1987). Constraint logic programming. Proc. of the fourteenth ACM Symposium on the Principles of Programming Languages (pp. 111–119).
Kapur, D. & Narendran, P. (1986). NP-completeness of the set unification and matching problems. Proc. of 8th Conference on Automated Deduction (Vol. 230, pp. 489–495). Springer Verlag.
Karalic, A. (1995). First order regression. Ph.D. Thesis, Institut Josef Stefan, Ljubljana, Slovenia.
Khardon, R. & Roth, D. (1997). Learning to reason. Jal of the ACM, 44(5), 697–725.
King, R. D., Srinivasan, A., & Sternberg, M. J. E. (1995). Relating chemical activity to structure: an examination of ILP successes. New Gen. Comput., 13, 411–433.
Kivinen, J. & Mannila, H. (1995). Approximate inference of functional dependencies from relations. Theoretical Computer Science, 149, 129–149.
Kodratoff, Y. & Ganascia, J.-G. (1986). Improving the generalization step in learning. In R. S. Michalski, J. G. Carbonell, & T. M. Mitchell (Eds.), Machine learning: an artificial intelligence approach (Vol. 2, pp. 215–244). Morgan Kaufmann.
Kohavi, R. & John, G. H. (1995). Automatic parameter selection by minimizing estimated error. In A. Prieditis, & S. Russell (Eds.), Proc. of the 12th ICML (pp. 304–312). Morgan Kaufmann.
Lavrač, N. & Džeroski, S. (1994). Inductive logic programming: techniques and applications. Ellis Horwood.
Lavrač, N., Gamberger, D., & Džeroski, S. (1995). An approach to dimensionality reduction in learning from deductive databases. In L. de Raedt (Ed.), Proc. of Inductive Logic Programming-95 (pp. 337–354). Katholieke Universiteit Leuven.
Lloyd, J. W. (1987). Foundations of logic programming. 2nd edn., Springer-Verlag.
Lloyd, J.W. (1999). Programming in an integrated functional and logic language. Journal of Functional and Logic Programming, March 1999.
Marquis, P. (1995). Knowledge compilation using prime implicants. Proc. of IJCAI-95 (pp. 837–843). Morgan Kaufmann.
Michalski, R. S. (1983). A theory and methodology of inductive learning. In R. S. Michalski, J. G. Carbonell, & T. M. Mitchell (Eds.), Machine learning: an artificial intelligence approach (Vol. 1, pp. 83–134). Morgan Kaufmann.
Mitchell, T. M. (1982). Generalization as search. Artificial Intelligence, 18, 203–226.
Mitchell, T. M. (1997). Machine learning. McGraw Hill.
Mitchell, D., Selman, B., & Levesque, H. (1992). Hard and easy distributions of SAT problems. Proc. of AAAI-92 (pp. 459–465). AAAI Press.
Muggleton, S. (1995). Inverse entailment and PROGOL. New Gen. Comput., 13, 245–286.
Muggleton, S. & De Raedt, L. (1994). Inductive logic programming: Theory and methods. Journal of Logic Programming, 19, 629–679.
Nédellec, C., Rouveirol, C., Adé, H., Bergadano, F., & Tausend, B. (1996). Declarative bias in ILP. In L. de Raedt (Ed.), Advances in ILP (pp. 82–103). IOS Press.
Norton, S. W. & Hirsh, H. (1993). Learning DNF via probabilistic evidence combination. In P. Utgoff (Ed.), Proc. of the 10th ICML (pp. 220–227). Morgan Kaufmann.
Patel-Schneider, P. F. (1990). A decidable first-order logic for knowledge representation. Journal of Automated Reasoning, 6, 361–388.
Pazzani, M. & Kibler, D. (1992). The role of prior knowledge in inductive learning. Machine Learning, 9, 54–97.
Plotkin, G. (1970). A note on inductive generalization. Machine intelligence (Vol. 5, pp. 153–163). Edinburgh University Press.
Quinlan, J. R. (1990). Learning logical definition from relations. Machine Learning, 5, 239–266.
Reiter, R. & De Kleer, J. (1987). Foundations of assumption-based truth maintenance systems: preliminary report. Proc. of AAAI-87 (pp. 183–188). AAAI Press.
Rissanen, J. (1978). Modelling by shortest data description. Automatica, 14, 465–471.
Rouveirol, C. (1994). Flattening and saturation: Two representation changes for generalisation. Machine Learning, 14, 219–232.
Russell, S. & Norwig, A. (1995). Artificial intelligence, a modern approach. Prentice Hall.
Sebag, M. (1994). Using constraints to building version spaces. In L. De Raedt & F. Bergadano (Eds.), Proc. of ECML-94, European conference on machine learning (pp. 257–271). Springer Verlag.
Sebag, M. (1995). 2nd order understandability of disjunctive version spaces. In Workshop on machine learning and comprehensibility, IJCAI-95. Report LRI, Université Paris-Sud, http://www.lri.fr/cn/wshp-ijcai.
Sebag, M. (1996). Delaying the choice of bias: A disjunctive version space approach. In L. Saitta (Ed.), Proc. of the 13th ICML (pp. 444–452). Morgan Kaufmann.
Sebag, M. & Rouveirol, C. (1997). Tractable induction and classification in FOL. Proc. of IJCAI-97 (pp. 888–892). Morgan Kaufmann.
Selman, B. & Kautz, H. (1996). Knowledge compilation and theory approximation. Jal of theACM, 43(2), 193–224.
Simon, H. (1982). Models of bounded rationality. MIT Press: Cambridge.
Smirnov, E. N. & Braspenning, P. J. (1998). Version space learning with instance-based boundary sets. H. Prade (Ed.), Proc. of European Conf. on Artificial Intelligence. John Wiley and Sons.
Smith, B. D. & Rosembloom, P. S. (1990). Incremental non-backtracking focusing: A polynomially bounded generalization algorithm for version space. Proc. of AAAI-90 (pp. 848–853). Morgan Kaufmann.
Srinivasan, A. & Muggleton, S. (1995). Comparing the use of background knowledge by two ILP systems. In L. de Raedt (Ed.), Proc. of Inductive Logic Programming-95. Katholieke Universiteit Leuven.
Srinivasan, A., Muggleton, S. H., Sternberg, M. J. E., & King, R. D. (1996). Theories for mutagenicity: a study in first order and feature-based induction. Artificial Intelligence, 85, 277–299.
Srinivasan, A. (1997). The predictive toxicology evaluation challenge. Proc. of IJCAI-97 (pp. 4–8). Morgan Kaufmann.
Zilberstein, S. (1996). Using any-time algorithms in intelligent systems. AI Magazine, 17(3), 73–83.
Zucker, J.-D. & Ganascia, J.-G. (1996). Representation changes for efficient learning in structural domains. In L. Saitta (Ed.), Proc. of the 13th ICML (pp. 543–551). Morgan Kaufmann.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Sebag, M., Rouveirol, C. Resource-bounded Relational Reasoning: Induction and Deduction Through Stochastic Matching. Machine Learning 38, 41–62 (2000). https://doi.org/10.1023/A:1007629922420
Issue Date:
DOI: https://doi.org/10.1023/A:1007629922420