Skip to main content
SpringerLink
Log in

Framework for Benchmarking Rule-Based Inference Engines

  • Conference paper
  • First Online:
Book cover Artificial Intelligence and Soft Computing (ICAISC 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10246))

Included in the following conference series:

Abstract

Rule-based systems constitute the state of the art solutions in the area of artificial intelligence. They provide fast, human readable and self explanatory mechanism for encoding knowledge. Due to large popularity of rules, dozens of inference engines were developed over last few decades. They differ in the reasoning efficiency depending on many factors such as model characteristics or deployment platform. Therefore, picking a reasoning engine that best fits the requirement of the system becomes a non-trivial task. The primary objective of the work presented in this paper was to provide a fully automated framework for benchmarking rule-based reasoning engines.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The full regulation text is available online. See: http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.L_.2016.119.01.0001.01.ENG&toc=OJ:L:2016:119:TOC.

  2. 2.

    See: http://clipsrules.sourceforge.net.

  3. 3.

    See: http://herzberg.ca.sandia.gov.

  4. 4.

    See: http://jruleengine.sourceforge.net.

  5. 5.

    See: http://www.easyrules.org.

  6. 6.

    See: ftp://ftp.cs.utexas.edu/pub/ops5-benchmark-suite.

  7. 7.

    See: http://wiki.ruleml.org/index.php/RuleML_Home.

  8. 8.

    See: http://glados.kis.agh.edu.pl/doku.php?id=pub:software:hwed:start.

  9. 9.

    The complete documentation of the generation process, with sample models used for benchmarks in this paper, can be found on the project website: https://bitbucket.org/sbobek/xtt-generator.

  10. 10.

    See: https://bitbucket.org/pm1234/xtt-generator.

  11. 11.

    The translation module source code can be found on the project website: https://bitbucket.org/sbobek/hmr-converter.

  12. 12.

    See: https://bitbucket.org/sbobek/subito.

  13. 13.

    GNU Octave is a high-level interpreted language, primarily intended for numerical computations. See: https://www.gnu.org/software/octave.

References

  1. Adrian, W.T., Bobek, S., Nalepa, G.J., Kaczor, K., Kluza, K.: How to reason by HeaRT in a semantic knowledge-based wiki. In: Proceedings of the 23rd IEEE International Conference on Tools with Artificial Intelligence, ICTAI 2011, Boca Raton, Florida, USA, pp. 438–441, November 2011. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6103361&tag=1

  2. Bobadilla, J., Ortega, F., Hernando, A., Gutiérrez, A.: Recommender systems survey. Knowl. Based Syst. 46, 109–132 (2013). http://www.sciencedirect.com/science/article/pii/S0950705113001044

  3. Bobek, S.: Methods for modeling self-adaptive mobile context-aware sytems. Ph.D. thesis, AGH University of Science and Technology, April 2016. Supervisor: G.J. Nalepa

    Google Scholar 

  4. Bobek, S., Nalepa, G.J.: Uncertain context data management in dynamic mobile environments. Future Gener. Comput. Syst. 66, 110–124 (2017). http://www.sciencedirect.com/science/article/pii/S0167739X1630187X

  5. Bobek, S., Nalepa, G.J., Ślażyński, M.: Challenges for migration of rule-based reasoning engine to a mobile platform. In: Dziech, A., Czyżewski, A. (eds.) MCSS 2014. CCIS, vol. 429, pp. 43–57. Springer, Cham (2014). doi:10.1007/978-3-319-07569-3_4

    Chapter  Google Scholar 

  6. Bock, J., Haase, P., Ji, Q., Volz, R.: Benchmarking OWL reasoners. In: van Harmelen, F., Herzig, A., Hitzler, P., Lin, Z., Piskac, R., Qi, G. (eds.) Proceedings of the ARea 2008 Workshop, vol. 350. CEUR Workshop Proceedings, June 2008. http://ceur-ws.org

  7. Boley, H., Paschke, A., Shafiq, O.: RuleML 1.0: the overarching specification of web rules. In: Dean, M., Hall, J., Rotolo, A., Tabet, S. (eds.) RuleML 2010. LNCS, vol. 6403, pp. 162–178. Springer, Heidelberg (2010). doi:10.1007/978-3-642-16289-3_15. http://dx.doi.org/10.1007/978-3-642-16289-3_15

    Chapter  Google Scholar 

  8. Brant, D., Grose, T., Lofaso, B., Miranker, D.: Effects of database size on rule system performance: five case studies. In: Proceedings of the 17th International Conference on Very Large Data Bases (VLDB) (1991)

    Google Scholar 

  9. Bratko, I.: Prolog Programming for Artificial Intelligence, 3rd edn. Addison Wesley, Redwood City (2000)

    MATH  Google Scholar 

  10. Dey, A.K.: Providing architectural support for building context-aware applications. Ph.D. thesis, Atlanta, GA, USA (2000). aAI9994400

    Google Scholar 

  11. Forgy, C.: Rete: a fast algorithm for the many patterns/many objects match problem. Artif. Intell. 19(1), 17–37 (1982)

    Article  Google Scholar 

  12. Giarratano, J.C., Riley, G.D.: Expert Systems. Thomson, Toronto (2005)

    Google Scholar 

  13. Goodman, B., Flaxman, S.: EU regulations on algorithmic decision-making and a “right to explanation” (2016). arXiv:1606.08813. Comment: Presented at 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016), New York, NY

  14. Hanson, E.N., Hasan, M.S.: Gator: an optimized discrimination network for active database rule condition testing. Technical report 93–036, CIS Department University of Florida, December 1993

    Google Scholar 

  15. Jang, J.H., Yang, S.H.: Development of the rule-based inference engine for the advanced context-awareness. Int. J. Smart Home 9(4), 195–202 (2015)

    Article  Google Scholar 

  16. Kaczor, K.: Knowledge formalization methods for semantic interoperability in rule bases. Ph.D. thesis, AGH University of Science and Technology, February 2015. Supervisor: G.J. Nalepa

    Google Scholar 

  17. Kaczor, K.: Practical approach to interoperability in production rule bases with Subito. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2015. LNCS, vol. 9120, pp. 637–648. Springer, Cham (2015). doi:10.1007/978-3-319-19369-4_56

    Chapter  Google Scholar 

  18. Ligęza, A.: Logical foundations for knowledge-based control systems - knowledge representation, reasoning and theoretical properties. Sci. Bull. AGH Autom. 63(1529), 144 (1993). Kraków

    MATH  Google Scholar 

  19. Lim, B.Y., Dey, A.K.: Investigating intelligibility for uncertain context-aware applications. In: Proceedings of the 13th International Conference on Ubiquitous Computing, UbiComp 2011, New York, NY, USA, pp. 415–424 (2011). http://doi.acm.org/10.1145/2030112.2030168

  20. Mahmud, U., Javed, M.Y.: Context inference engine (CIE): inferring context. Int. J. Adv. Pervasive Ubiquitous Comput. 4(3), 13–41, July 2012. http://dx.doi.org/10.4018/japuc.2012070102

  21. Miranker, D.P., Lofaso, B.J.: The organization and performance of a treat-based production system compiler. IEEE Trans. Knowl. Data Eng. 3(1), 3–10 (1991)

    Article  Google Scholar 

  22. Miranker, D.P.: TREAT: a better match algorithm for AI production systems, long version. Technical report 87–58, University of Texas, July 1987

    Google Scholar 

  23. Nalepa, G.J., Bobek, S., Ligęza, A., Kaczor, K.: HalVA - rule analysis framework for XTT2 Rules. In: Bassiliades, N., Governatori, G., Paschke, A. (eds.) RuleML 2011. LNCS, vol. 6826, pp. 337–344. Springer, Heidelberg (2011). doi:10.1007/978-3-642-22546-8_27. http://www.springerlink.com/content/c276374nh9682jm6/

    Chapter  Google Scholar 

  24. Nalepa, G.J., Ligęza, A., Kaczor, K.: Overview of knowledge formalization with XTT2 Rules. In: Bassiliades, N., Governatori, G., Paschke, A. (eds.) RuleML 2011. LNCS, vol. 6826, pp. 329–336. Springer, Heidelberg (2011). doi:10.1007/978-3-642-22546-8_26

    Chapter  Google Scholar 

  25. Nalepa, G.J.: Loki–semantic wiki with logical knowledge representation. In: Nguyen, N.T. (ed.) Transactions on Computational Collective Intelligence III. LNCS, vol. 6560, pp. 96–114. Springer, Heidelberg (2011). doi:10.1007/978-3-642-19968-4_5. http://www.springerlink.com/content/y91w134g03344376/

    Chapter  Google Scholar 

  26. Nalepa, G.J., Bobek, S.: Rule-based solution for context-aware reasoning on mobile devices. Comput. Sci. Inf. Syst. 11(1), 171–193 (2014)

    Article  Google Scholar 

  27. Nalepa, G.J., Kluza, K., Kaczor, K.: Proposal of an inference engine architecture for business rules and processes. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013. LNCS, vol. 7895, pp. 453–464. Springer, Heidelberg (2013). doi:10.1007/978-3-642-38610-7_42. http://www.springer.com/computer/ai/book/978-3-642-38609-1

    Chapter  Google Scholar 

  28. Ostermayer, L.: Seamless cooperation of java and prolog for rule-based software development. In: Proceedings of the RuleML 2015 Challenge, the Special Track on Rule-based Recommender Systems for the Web of Data, the Special Industry Track and the RuleML 2015 Doctoral Consortium hosted by the 9th International Web Rule Symposium (RuleML 2015), Berlin, Germany, 2–5 August 2015 (2015). http://ceur-ws.org/Vol-1417/paper2.pdf

  29. Pan, Z.: Benchmarking dl reasoners using realistic ontologies. In: Grau, B.C., Horrocks, I., Parsia, B., Patel-Schneider, P.F. (eds.) OWLED. CEUR Workshop Proceedings, vol. 188. CEUR-WS.org (2005). http://dblp.uni-trier.de/db/conf/owled/owled2005.html#Pan05

  30. Weert, P.V.: Efficient lazy evaluation of rule-based programs. IEEE Trans. Knowl. Data Eng. 22(11), 1521–1534 (2010)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. AGH University of Science and Technology, al. Mickiewicza 30, 30-059, Krakow, Poland

    Szymon Bobek & Piotr Misiak

Authors
  1. Szymon Bobek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Piotr Misiak
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Szymon Bobek .

Editor information

Editors and Affiliations

  1. Częstochowa University of Technology, Częstochowa, Poland

    Leszek Rutkowski

  2. Częstochowa University of Technology, Częstochowa, Poland

    Marcin Korytkowski

  3. Częstochowa University of Technology, Częstochowa, Poland

    Rafał Scherer

  4. AGH University of Science and Technology, Kraków, Poland

    Ryszard Tadeusiewicz

  5. University of California, Berkeley, California, USA

    Lotfi A. Zadeh

  6. University of Louisville, Louisville, Kentucky, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Bobek, S., Misiak, P. (2017). Framework for Benchmarking Rule-Based Inference Engines. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L., Zurada, J. (eds) Artificial Intelligence and Soft Computing. ICAISC 2017. Lecture Notes in Computer Science(), vol 10246. Springer, Cham. https://doi.org/10.1007/978-3-319-59060-8_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-59060-8_36

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-59059-2

  • Online ISBN: 978-3-319-59060-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation