Abstract
Companies spend significant efforts on testing their products to achieve a sufficient quality level. This paper presents results from evaluating the quality impact of implementing a framework for component-level test automation and Test-Driven Development. The evaluation comprised six projects for two products at a software development department at Ericsson. The paper suggests how an existing measurement approach can be used for evaluating the quality impact of improvements in early phases, i.e. by classifying faults reported on released products after which phase they should have been caught in. Based on this measurement approach, the evaluation determined that the ratio of reported faults in the released products decreased significantly after implementing the framework. That is, the ratio of faults belonging to component-level testing decreased from between 60-70 percent to less than 20 percent in the two studied products.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bassin, K.A., Kratschmer, T., Santhanam, P.: Evaluating Software Objectively. IEEE Software 15(6), 66–74 (1998)
Beck, K.: Test Driven Development - by example. Addison-Wesley, Boston, MA (2003)
Beizer, B.: Software Testing Techniques, 2nd edn. Van Nostrand Reinhold Company, New York (1990)
Bhat, T., Nagappan, N.: Evaluating the Efficacy of Test-Driven Development: Industrial Case Studies. In: Proceedings of the 5th International Symposium on Empirical Software Engineering, pp. 356–363. ACM Press, New York (2006)
Bundell, G.A., Lee, G., Morris, J., Parker, K., Peng, L.: A software component verification tool. In: Proceedings of International Conference on Software Methods and Tools, pp. 137–146. IEEE Computer Soc., Los Alamitos
Chillarege, R., Bhandari, I., Chaar, M., Halliday, D., Moebus, B., Ray, B., Wong, M.Y.: Orthogonal Defect Classification-A concept for In-Process Measurement. IEEE Transactions on Software Engineering 18, 943–956 (1992)
Cockburn, A.: Agile Software Development. Addison-Wesley, Boston, MA (2002)
Crispin, L.: Driving Software Quality: How Test-Driven Development Impacts Software Quality. IEEE Software 23(6), 70–71 (2006)
Damm, L.-O., Lundberg, L.: Introducing Test Automation and Test-Driven Development: An Experience Report. In: Proceedings of the International Workshop on Test and Analysis of Component-Based Systems. Electronic Notes in Theoretical Computer Science, vol. 316, pp. 3–15. Elsevier, Amsterdam (2004)
Damm, L.-O., Lundberg, L.: Identification of Test Process Improvements by Combining ODC Triggers and Faults-Slip-Through. In: Proceedings of the 4th International Symposium on Empirical Software Engineering, pp. 152–161. IEEE, Los Alamitos (2005)
Damm, L.-O., Lundberg, L.: Results from Introducing Component-Level Test Automation and Test-Driven Development. Journal of Systems and Software 79, 1001–1014 (2006)
Damm, L.-O., Lundberg, L., Wohlin, C.: Faults-Slip-Through - A Concept for Measuring the Efficiency of the Test Process. Wiley Journal of Software: Process Improvement and Practice 11, 47–59 (2006)
Erdogmus, H., Morisio, M.: On the Effectiveness of the Test-First Approach to Programming. IEEE Transactions on Software Engineering 31(3), 226–237 (2005)
Fenton, N., Pfleeger, S.L.: Software Metrics: A Rigorous Approach. PWS Publishing Company (1997)
Gao, J., Tsao, J., Wu, Y.: Testing and Quality Assurance for Component-Based Software. Artech House Publishers (2003)
George, B., Williams, L.: A structured experiment of test-driven development. Information and Software Technology 46, 337–342 (2004)
Geras, A., Smith, M.R., Miller, J.: A Prototype Empirical Evaluation of Test Driven Development. In: Proceedings of the 10th IEEE International Software Metrics Symposium, pp. 405–416. IEEE Computer Society, Los Alamitos (2004)
Harrold, J.: Testing: a roadmap. In: International Conference on Software Engineering, pp. 61–72. ACM, New York (2000)
Hevner, A.R.: Phase Containment for Software Quality Improvement. Information and Software Technology 39, 867–877 (1997)
Howles, T., Daniels, S.: Widespread Effects of Defects. Quality Progress 36(8), 58–62 (2003)
Maximilien, E.M., Williams, L.: Assessing Test-Driven Development at IBM. In: Proceedings of the 25th International Conference on Software Engineering, pp. 564–569. IEEE Computer Soc. Press, Los Alamitos (2003)
Robson, C.: Real World Research, 2nd edn. Blackwell Publishers, Oxford, UK (2002)
Runeson, P., Andersson, C., Thelin, T., Andrews, A., Berling, T.: What Do We Know about Defect Detection Methods? IEEE Software 23(3), 82–90 (2006)
Shull, F., Basili, V., Boehm, B., Brown, W., Costa, P., Lindwall, M., Port, D., Rus, I., Tesoriero, R., Zelkowitz, M.: What We Have Learned About Fighting Defects. In: Proceedings of the Eight IEEE Symposium on Software Metrics, pp. 249–258. IEEE, Los Alamitos (2002)
Teiniker, E., Mitterdorfer, S., Johnson, L.M., Johnson, L.M., Kreiner, C., Kovacs, Z., Weiss, R.: A Test-Driven Component Development Framework Based On The Corba Component Model. In: Proceedings of the 27th Annual International Computer Software and Applications Conference, pp. 400–405. IEEE, Los Alamitos (2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Damm, LO., Lundberg, L. (2007). Quality Impact of Introducing Component-Level Test Automation and Test-Driven Development. In: Abrahamsson, P., Baddoo, N., Margaria, T., Messnarz, R. (eds) Software Process Improvement. EuroSPI 2007. Lecture Notes in Computer Science, vol 4764. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75381-0_17
Download citation
DOI: https://doi.org/10.1007/978-3-540-75381-0_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74765-9
Online ISBN: 978-3-540-75381-0
eBook Packages: Computer ScienceComputer Science (R0)