Robert M. Hierons
Skip Abstract Section
Abstract
Many systems interact with their environment at distributed interfaces (ports) and sometimes it is not possible to place synchronised local testers at the ports of the system under test (SUT). There are then two main approaches to testing: having independent local testers or a single centralised tester that interacts asynchronously with the SUT. The power of using independent testers has been captured using implementation relation dioco. In this article, we define implementation relation diococ for the centralised approach and prove that dioco and diococ are incomparable. This shows that the frameworks detect different types of faults and so we devise a hybrid framework and define an implementation relation diocos for this. We prove that the hybrid framework is more powerful than the distributed and centralised approaches. We then prove that the Oracle problem is NP-complete for diococ and diocos but can be solved in polynomial time if we place an upper bound on the number of ports. Finally, we consider the problem of deciding whether there is a test case that is guaranteed to force a finite state model into a particular state or to distinguish two states, proving that both problems are undecidable for the centralised and hybrid frameworks.
- F. Aarts and F. W. Vaandrager. 2010. Learning I/O Automata. In Proceedings of the 21th International Conference on Concurrency Theory (CONCUR 2010). Lecture Notes in Computer Science, Vol. 6269. Springer, 71--85. Google Scholar
Digital Library
- R. Alur, C. Courcoubetis, and M. Yannakakis. 1995. Distinguishing tests for nondeterministic and probabilistic machines. In Proceedings of the 27th ACM Symposium on Theory of Computing. 363--372. Google ScholarDigital Library
- E. Brinksma, L. Heerink, and J. Tretmans. 1998. Factorized test generation for multi-input/output transition systems. In Proceedings of the 11th IFIP International Workshop on Testing Communicating Systems (IWTCS) (IFIP Conference Proceedings), vol. 131. Kluwer, 67--82. Google ScholarDigital Library
- L. Cacciari and O. Rafiq. 1999. Controllability and observability in distributed testing. Inf. Softw. Tech. 41, 11--12, 767--780.Google ScholarCross Ref
- J. Chen, R. M. Hierons, and H. Ural. 2005. Resolving observability problems in distributed test architectures. In Proceedings of the Formal Techniques for Networked and Distributed Systems (FORTE 2005). Lecture Notes in Computer Science, Vol. 3731. Springer, 219--232. Google ScholarDigital Library
- W.-H. Chen and H. Ural. 1995. Synchronizable checking sequences based on multiple UIO sequences. IEEE/ACM Trans. Netw. 3, 152--157. Google ScholarDigital Library
- T. S. Chow. 1978. Testing software design modelled by finite state machines. IEEE Trans. Softw. Eng. 4, 178--187. Google ScholarDigital Library
- M. D. Davis and E. J. Weyuker. 1993. Computability, Complexity and Languages. Academic Press. Google ScholarDigital Library
- E. D. Demaine and R. A. Hearn. 2008. Constraint logic: A uniform framework for modeling computation as games. In Proceedings of the 23rd Annual IEEE Conference on Computational Complexity (CCC 2008). 149--162. Google ScholarDigital Library
- R. Dssouli and G. von Bochmann. 1985. Error detection with multiple observers. In Protocol Specification, Testing and Verification V. Elsevier Science (North Holland), 483--494. Google ScholarDigital Library
- R. Dssouli and G. von Bochmann. 1986. Conformance testing with multiple observers. In Protocol Specification, Testing and Verification VI. Elsevier Science (North Holland), 217--229.Google ScholarDigital Library
- A. En-Nouaary. 2013. A test purpose-based approach for testing timed input output automata. J. Softw. Test. Verif. Reliab. 23, 1, 53--76. Google ScholarDigital Library
- E. Farchi, A. Hartman, and S. S. Pinter. 2002. Using a model-based test generator to test for standard conformance. IBM Syst. J. 41, 1, 89--110. Google ScholarDigital Library
- R. W. Floyd. 1964. New Proofs and Old Theorems in Logic and Formal Linguistics. Computer Associated Inc., Wakefield, MA.Google Scholar
- M.-C. Gaudel. 1995. Testing can be formal too. In Proceedings of the 6th International Joint Conference CAAP/FASE Theory and Practice of Software Development (TAPSOFT'95). Lecture Notes in Computer Science, Vol. 915. Springer, 82--96. Google ScholarDigital Library
- G. Gonenc. 1970. A method for the design of fault detection experiments. IEEE Trans. Comput. 19, 551--558. Google ScholarDigital Library
- W. Grieskamp. 2006. Multi-paradigmatic model-based testing. In Formal Approaches to Software Testing and Runtime Verification (FATES/RV 2006). Lecture Notes in Computer Science, Vol. 4262. Springer, 1--19. Google ScholarDigital Library
- W. Grieskamp, N. Kicillof, K. Stobie, and V. A. Braberman. 2011. Model-based quality assurance of protocol documentation: tools and methodology. Softw. Test. Verif. Reliab. 21, 1, 55--71. Google ScholarDigital Library
- L. Heerink and J. Tretmans. 1997. Refusal testing for classes of transition systems with inputs and outputs. In Proceedings of the Joint International Conference on Formal Description Techniques and Protocol Specification, Testing and Verification (FORTE X/PSTV XVII) (IFIP Conference Proceedings), Vol. 107. Chapman & Hall, 23--38. Google ScholarDigital Library
- F. C. Hennie. 1964. Fault-detecting experiments for sequential circuits. In Proceedings of the 5th Annual Symposium on Switching Circuit Theory and Logical Design. ACM, New York, 95--110. Google ScholarDigital Library
- R. M. Hierons. 2010. Reaching and distinguishing states of distributed systems. SIAM J. Comput. 39, 8, 3480--3500. Google ScholarDigital Library
- R. M. Hierons. 2012a. The complexity of asynchronous model based testing. Theoret. Comput. Sci. 451, 70--82. Google ScholarDigital Library
- R. M. Hierons. 2012b. Oracles for distributed testing. IEEE Trans. Softw. Eng. 38, 3, 629--641. Google ScholarDigital Library
- R. M. Hierons. 2013. Implementation relations for testing through asynchronous channels. Comput. J. 56, 11, 1305--1319.Google ScholarCross Ref
- R. M. Hierons, M. G. Merayo, and M. Núñez. 2012. Implementation relations and test generation for systems with distributed interfaces. Distrib. Comput. 25, 1, 35--62.Google ScholarCross Ref
- R. M. Hierons and H. Ural. 2008a. Checking sequences for distributed test architectures. Distrib. Comput. 21, 3, 223--238.Google ScholarDigital Library
- R. M. Hierons and H. Ural. 2008b. The effect of the distributed test architecture on the power of testing. Comput. J. 51, 4, 497--510. Google ScholarDigital Library
- J. Huo and A. Petrenko. 2004. On testing partially specified IOTS through lossless queues. In Proceedings of the 16th IFIP International Conference on the Testing of Communicating Systems (TestCom 2004). Lecture Notes in Computer Science, Vol. 2978. Springer, 76--94.Google Scholar
- I. Hwang, A. R. Cavalli, M. Lallali, and D. Verchère. 2012. Applying formal methods to PCEP: An industrial case study from modeling to test generation. Softw. Test. Verif. Reliab. 22, 5, 343--361. Google ScholarDigital Library
- ISO/IEC. 1995. Information technology - Opens Systems Interconnection, 9646 Parts 1-7. ISO/IEC.Google Scholar
- C. Jard, T. Jéron, H. Kahlouche, and C. Viho. 1998. Towards automatic distribution of testers for distributed conformance testing. In Proceedings of the TC6 WG6.1 Joint International Conference on Formal Description Techniques and Protocol Specification, Testing and Verification (FORTE 1998) (IFIP Conference Proceedings), Vol. 135. Kluwer, 353--368. Google ScholarDigital Library
- G.-V. Jourdan, H. Ural, and H. Yenigün. 2006. Minimizing coordination channels in distributed testing. In Proceedings of the Joint International Conference on Formal Techniques for Networked and Distributed Systems (FORTE 2006). Lecture Notes in Computer Science, Vol. 4229. Springer, 451--466. Google ScholarDigital Library
- D. Lee and M. Yannakakis. 1994. Testing finite-state machines: State identification and verification. IEEE Trans. Comput. 43, 3, 306--320. Google ScholarDigital Library
- D. Lee and M. Yannakakis. 1996. Principles and methods of testing finite-state machines-A survey. Proc. IEEE 84, 8, 1089--1123.Google ScholarCross Ref
- Z. Li, J. Wu, and X. Yin. 2004. Testing multi input/output transition system with all-observer. In Proceedings of the 16th IFIP International Conference on Testing of Communicating Systems (TestCom 2004). Lecture Notes in Computer Science, Vol. 2978. Springer, 95--111.Google Scholar
- T. Miller and P. A. Strooper. 2012. A case study in model-based testing of specifications and implementations. Softw. Test. Verif. Reliab. 22, 1, 33--63.Google ScholarCross Ref
- E. F. Moore. 1956. Gedanken-Experiments. In Automata Studies, C. Shannon and J. McCarthy (Eds.). Princeton University Press.Google Scholar
- N. Noroozi, R. Khosravi, M. R. Mousavi, and T. A. C. Willemse. 2011. Synchronizing asynchronous conformance testing. In Proceedings of the 9th International Conference on Software Engineering and Formal Methods (SEFM 2011). Lecture Notes in Computer Science, Vol. 7041. Springer, 334--349. Google ScholarDigital Library
- G. L. Peterson and J. H. Reif. 1979. Multiple-person alternation. In Proceedings of the 20th Annual Symposium on Foundations of Computer Science (FOCS 79). IEEE, 348--363. Google ScholarDigital Library
- A. Petrenko, S. Boroday, and R. Groz. 2004. Confirming configurations in EFSM testing. IEEE Trans. Softw. Eng. 30, 1 (2004), 29--42. Google ScholarDigital Library
- A. Petrenko, N. Yevtushenko, and J. Huo. 2003. Testing transition systems with input and output testers. In Proceedings of the 15th IFIP International Conference on Testing of Communicating Systems (TestCom 2003). Lecture Notes in Computer Science, Vol. 2644. Springer, 129--145. Google ScholarDigital Library
- O. Rafiq and L. Cacciari. 2003. Coordination algorithm for distributed testing. J. Supercomput. 24, 2, 203--211. Google ScholarDigital Library
- B. Sarikaya and G. von Bochmann. 1984. Synchronization and specification issues in protocol testing. IEEE Trans. Commun. 32 (April 1984), 389--395.Google ScholarCross Ref
- T. J. Schaefer. 1978. The complexity of satisfiability problems. In Proceedings of the 10th Annual ACM Symposium on Theory of Computing (STOC). 216--226. Google ScholarDigital Library
- A. Simão and A. Petrenko. 2011. Generating asynchronous test cases from test purposes. Inf. Softw. Tech. 53, 1252--1262.Google ScholarCross Ref
- J. Swigart and S. Lall. 2011. Optimal controller synthesis for a decentralized two-player system with partial output feedback. In Proceedings of the American Control Conference (ACC), 2011. 317--323.Google Scholar
- L. H. Tahat, B. Korel, M. Harman, and H. Ural. 2012. Regression test suite prioritization using system models. Softw. Test. Verif. Reliab. 22, 7, 481--506. Google ScholarDigital Library
- K.-C. Tai and Y.-C. Young. 1998. Synchronizable test sequences of finite state machines. Comput. Netw. ISDN Syst. 30, 12, 1111--1134. Google ScholarDigital Library
- R. E. Tarjan. 1972. Depth-first search and linear graph algorithms. SIAM J. Comput. 1, 2, 146--160.Google ScholarDigital Library
- J. Tretmans. 1996. Conformance testing with labelled transitions systems: Implementation relations and test generation. Comput. Netw. ISDN Syst. 29, 1, 49--79. Google ScholarDigital Library
- J. Tretmans. 2008. Model based testing with labelled transition systems. In Proceedings of the Formal Methods and Testing. Lecture Notes in Computer Science, Vol. 4949. Springer, 1--38. Google ScholarDigital Library
- M. Veanes and N. Bjørner. 2010. Alternating simulation and IOCO. In Proceedings of the 22nd IFIP WG 6.1 International Conference on Testing Software and Systems (ICTSS). Lecture Notes in Computer Science, Vol. 6435. Springer, 47--62. Google ScholarDigital Library
- M. Weiglhofer and F. Wotawa. 2009. Asynchronous input-output conformance testing. In Proceedings of the 33rd Annual IEEE International Computer Software and Applications Conference (COMPSAC 2009). IEEE Computer Society, 154--159. Google ScholarDigital Library
- T. C. Yang, J. H. Zhang, and H. Yu. 1999. A new decentralised controller design method with application to power-system stabiliser design. Control Engineering Practice 7, 4, 537--545. DOI: http://dx.doi.org/10.1016/S0967-0661(99)00014-3Google ScholarCross Ref
Index Terms
- Combining Centralised and Distributed Testing
Recommendations
Generating Complete Controllable Test Suites for Distributed Testing
A test suite is m-complete for finite state machine (FSM) M if it distinguishes between M and all faulty FSMs with m states or fewer. While there are several algorithms that generate m-complete test suites, they cannot be directly used in distributed ...
Distributed Online Test Generation for Model-Based Testing
APSEC '13: Proceedings of the 2013 20th Asia-Pacific Software Engineering Conference (APSEC) - Volume 01In online model-based testing, test execution is interleaved with test generation. Test cases should be generated and executed with minimal delay, while still achieving targeted coverage criteria quickly. Extensive model analysis in such case is not ...
Distributed Testing of an Equipment-Level Interface Specification
ISORC '02: Proceedings of the Fifth IEEE International Symposium on Object-Oriented Real-Time Distributed ComputingA test suite for a key interface within a dimensional measuring system (coordinate measuring machine or CMM) is presented. The test suite consists of test procedures, test definitions, and various testing utilities. A real-time, distributed test ...
Comments