Antonia Bertolino
Emilio Cruciani
Breno Miranda
Roberto Verdecchia
ABSTRACT
We propose a novel scalable approach for testing non-testable programs denoted as ARMED testing. The approach leverages efficient Association Rules Mining algorithms to determine relevant implication relations among features and actions observed while the system is in operation. These relations are used as the specification of positive and negative tests, allowing for identifying plausible or suspicious behaviors: for those cases when oracles are inherently unknownable, such as in social testing, ARMED testing introduces the novel concept of testing for plausibility. To illustrate the approach we walk-through an application example.
- Rakesh Agrawal, Tomasz Imieliński, and Arun Swami. 1993. Mining Association Rules between Sets of Items in Large Databases. SIGMODRec. 22, 2 (June 1993), 207--216. Google Scholar
Digital Library
- Rakesh Agrawal, Ramakrishnan Srikant, et al. 1994. Fast algorithms for mining association rules. In Proc. 20th int. conf. very large data bases, VLDB, Vol. 1215. Citeseer, 487--499.Google ScholarDigital Library
- John Ahlgren, Maria Eugenia Berezin, Kinga Bojarczuk, Elena Dulskyte, Inna Dvortsova, Johann George, Natalija Gucevska, Mark Harman, Ralf Laemmel, Erik Meijer, et al. 2020. WES: Agent-based user interaction simulation on real infrastructure. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops. 276--284.Google ScholarDigital Library
- John Ahlgren, Maria Eugenia Berezin, Kinga Bojarczuk, Elena Dulskyte, Inna Dvortsova, Johann George, Natalija Gucevska, Mark Harman, Maria Lomeli, Erik Meijer, et al. 2021. Testing Web Enabled Simulation at Scale Using Metamorphic Testing. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 140--149. Google ScholarDigital Library
- John Ahlgren, Kinga Bojarczuk, Sophia Drossopoulou, Inna Dvortsova, Johann George, Natalija Gucevska, Mark Harman, Maria Lomeli, Simon M. M. Lucas, Erik Meijer, Steve Omohundro, Rubmary Rojas, Silvia Sapora, and Norm Zhou. 2021. Facebook's Cyber-Cyber and Cyber-Physical Digital Twins. In Evaluation and Assessment in Software Engineering (Trondheim, Norway) (EASE 2021). Association for Computing Machinery, New York, NY, USA, 1--9. Google ScholarDigital Library
- Earl T. Barr, Mark Harman, Phil McMinn, Muzammil Shahbaz, and Shin Yoo. 2015. The Oracle Problem in Software Testing: A Survey. IEEE Trans. Softw. Eng. 41, 5 (May 2015), 507--525. Google ScholarDigital Library
- Pan Bian et al. 2018. Nar-miner: Discovering negative association rules from code for bug detection. In ESEC/FSE. 411--422.Google Scholar
- Tsong Yueh Chen et al. 2018. Metamorphic Testing: A Review of Challenges and Opportunities. ACM Comput. Surv. 51, 1, Article 4 (Jan. 2018), 27 pages. Google ScholarDigital Library
- Junhua Ding, Dongmei Zhang, and Xin-Hua Hu. 2016. An application of metamorphic testing for testing scientific software. In Proceedings of the 1st International Workshop on Metamorphic Testing. 37--43.Google ScholarDigital Library
- Michael D Ernst, Jeff H Perkins, Philip J Guo, Stephen McCamant, Carlos Pacheco, Matthew S Tschantz, and Chen Xiao. 2007. The Daikon system for dynamic detection of likely invariants. Science of computer programming 69, 1-3 (2007), 35--45.Google Scholar
- Philippe Fournier-Viger, Jerry Chun-Wei Lin, Bay Vo, Tin Chi Truong, Ji Zhang, and Hoai Bac Le. 2017. A survey of itemset mining. WIREs Data Mining Knowl. Discov. 7, 4 (2017). Google ScholarCross Ref
- Jiawei Han, Jian Pei, and Yiwen Yin. 2000. Mining frequent patterns without candidate generation. ACM sigmod record 29, 2 (2000), 1--12.Google Scholar
- Kim Herzig and Nachiappan Nagappan. 2015. Empirically detecting false test alarms using association rules. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Vol. 2. IEEE, 39--48.Google ScholarCross Ref
- Benjamin Livshits and Thomas Zimmermann. 2005. Dynamine: finding common error patterns by mining software revision histories. ACM SIGSOFT Software Engineering Notes 30, 5 (2005), 296--305.Google ScholarDigital Library
- José María Luna, Philippe Fournier-Viger, and Sebastian Ventura. 2019. Frequent itemset mining: A 25 years review. WIREs Data Mining Knowl. Discov. 9, 6 (2019). Google ScholarCross Ref
- Manishankar Mandal, Chanchal K Roy, and Kevin A Schneider. 2014. Automatic ranking of clones for refactoring through mining association rules. In 2014 Software Evolution Week-IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering (CSMR-WCRE). IEEE, 114--123.Google ScholarCross Ref
- Bhabesh Nath, D. K. Bhattacharyya, and Ashish Ghosh. 2013. Incremental association rule mining: a survey. WIREs Data Mining Knowl. Discov. 3, 3 (2013), 157--169. Google ScholarCross Ref
- Krishna Patel and Robert M. Hierons. 2018. A Mapping Study on Testing Non-Testable Systems. Software Quality Journal 26, 4 (Dec. 2018), 1373--1413. Google ScholarDigital Library
- Gregory Piatetsky-Shapiro. 1991. Discovery, analysis, and presentation of strong rules. Knowledge discovery in databases (1991), 229--238.Google Scholar
- Sebastiaan Rothmann and Elize P Coetzer. 2003. The big five personality dimensions and job performance. SA Journal of Industrial Psychology 29, 1 (2003), 68--74.Google ScholarCross Ref
- Ashok Savasere et al. 1995. An efficient algorithm for mining association rules in large databases. Technical Report. Georgia Institute of Technology.Google Scholar
- David J Stillwell and Michal Kosinski. 2007. myPersonality database. https://www.psychometrics.cam.ac.uk/productsservices/mypersonality Accessed on 09/10/2021.Google Scholar
- Hannu Toivonen. 1996. Sampling Large Databases for Association Rules. In Proceedings of the 22th International Conference on Very Large Data Bases (VLDB '96). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 134--145.Google ScholarDigital Library
- Adriano Veloso, Wagner Meira Jr., Márcio de Carvalho, Bruno Pôssas, Srinivasan Parthasarathy, and Mohammed Javeed Zaki. 2002. Mining Frequent Itemsets in Evolving Databases. In Proceedings of the Second SIAM International Conference on Data Mining, Arlington, VA, USA, April 11-13, 2002, Robert L. Grossman, Jiawei Han, Vipin Kumar, Heikki Mannila, and Rajeev Motwani (Eds.). SIAM, 494--510. Google ScholarCross Ref
- Stanley Wasserman and Katherine Faust. 1994. Social Network Analysis: Methods and Applications. Cambridge University Press. Google ScholarCross Ref
- Elaine J Weyuker. 1982. On testing non-testable programs. Comput. J. 25, 4 (1982), 465--470.Google ScholarCross Ref
- Xiaoyuan Xie, Joshua WK Ho, Christian Murphy, Gail Kaiser, Baowen Xu, and Tsong Yueh Chen. 2011. Testing and validating machine learning classifiers by metamorphic testing. Journal of Systems and Software 84, 4 (2011), 544--558.Google ScholarDigital Library
- Mohammed Javeed Zaki. 2000. Scalable algorithms for association mining. IEEE transactions on knowledge and data engineering 12, 3 (2000), 372--390.Google ScholarDigital Library
- Wujie Zheng, Hao Ma, Michael R. Lyu, Tao Xie, and Irwin King. 2011. Mining Test Oracles of Web Search Engines. In Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE '11). IEEE Computer Society, USA, 408--411. Google ScholarDigital Library
Recommendations
Mining Negatives Association Rules Using Constraints
Pattern discovery techniques, such as association rule discovery is one of the fundamental problem in data mining. Usually the task is limited to positive rules of the form X Y when X and Y are susbets of items. To enlarge the knowledge discovery from ...
Testing web enabled simulation at scale using metamorphic testing
ICSE-SEIP '21: Proceedings of the 43rd International Conference on Software Engineering: Software Engineering in PracticeWe report on Facebook's deployment of MIA (Metamorphic Interaction Automaton). MIA is used to test Facebook's Web Enabled Simulation, built on a web infrastructure of hundreds of millions of lines of code. MIA tackles the twin problems of test flakiness ...
Empirically detecting false test alarms using association rules
ICSE '15: Proceedings of the 37th International Conference on Software Engineering - Volume 2Applying code changes to software systems and testing these code changes can be a complex task that involves many different types of software testing strategies, e.g. system and integration tests. However, not all test failures reported during code ...
Comments