Skip to main content
Springer Nature Link
Log in

Effects of Test-Driven Development: A Comparative Analysis of Empirical Studies

  • Conference paper
Software Quality. Model-Based Approaches for Advanced Software and Systems Engineering (SWQD 2014)

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 166))

Included in the following conference series:

Abstract

Test-driven development is a software development practice where small sections of test code are used to direct the development of program units. Writing test code prior to the production code promises several positive effects on the development process itself and on associated products and processes as well. However, there are few comparative studies on the effects of test-driven development. Thus, it is difficult to assess the potential process and product effects when applying test-driven development. In order to get an overview of the observed effects of test-driven development, an in-depth review of existing empirical studies was carried out. The results for ten different internal and external quality attributes indicate that test-driven development can reduce the amount of introduced defects and lead to more maintainable code. Parts of the implemented code may also be somewhat smaller in size and complexity. While maintenance of test-driven code can take less time, initial development may last longer. Besides the comparative analysis, this article sketches related work and gives an outlook on future research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Beck, K.: Test-Driven Development: By Example. Addison-Wesley (2003)

    Google Scholar 

  2. Turhan, B., Layman, L., Diep, M., Erdogmus, H., Shull, F.: How Effective is Test-Driven Development. In: Oram, A., Wilson, G. (eds.) Making Software: What Really Works, and Why We Believe It, pp. 207–219. O’Reilly (2010)

    Google Scholar 

  3. Jeffries, R., Melnik, G.: Guest Editors’ Introduction: TDD–The Art of Fearless Programming. IEEE Software 24(3), 24–30 (2007)

    Article  Google Scholar 

  4. Rafique, Y., Misic, V.: The Effects of Test-Driven Development on External Quality and Productivity: A Meta Analysis. IEEE Transactions on Software Engineering 39(6), 835–856 (2013)

    Article  Google Scholar 

  5. Desai, C., Janzen, D., Savage, K.: A Survey of Evidence for Test-Driven Development in Academia. SIGCSE Bulletin 40(2), 97–101 (2008)

    Article  Google Scholar 

  6. Runeson, P., Höst, M.: Guidelines for Conducting and Reporting Case Study Research in Software Engineering. Empirical Software Engineering 14, 131–164 (2009)

    Article  Google Scholar 

  7. Merriam, S.B.: Qualitative Research: A Guide to Design and Implementation. John Wiley & Sons (2009)

    Google Scholar 

  8. Torraco, R.J.: Writing Integrative Literature Reviews: Guidelines and Examples. Human Resource Development Review 4(3), 356–367 (2005)

    Article  Google Scholar 

  9. Kitchenham, B., Charters, S.: Guidelines for Performing Systematic Literature Reviews in Software Engineering. Technical Report Version 2.3, Keele University and University of Durham (July 2007)

    Google Scholar 

  10. Creswell, J.W.: Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage Publications (2009)

    Google Scholar 

  11. Mäkinen, S.: Driving Software Quality and Structuring Work Through Test-Driven Development. Master’s thesis, Department of Computer Science, University of Helsinki (October 2012)

    Google Scholar 

  12. Bhat, T., Nagappan, N.: Evaluating the Efficacy of Test-driven Development: Industrial Case Studies. In: Proceedings of the 2006 ACM/IEEE International Symposium on Empirical Software Engineering, ISESE 2006, pp. 356–363. ACM, New York (2006)

    Google Scholar 

  13. Canfora, G., Cimitile, A., Garcia, F., Piattini, M., Visaggio, C.A.: Evaluating Advantages of Test Driven Development: A Controlled Experiment with Professionals. In: Proceedings of the 2006 ACM/IEEE International Symposium on Empirical Software Engineering, ISESE 2006, pp. 364–371. ACM, New York (2006)

    Google Scholar 

  14. Dogša, T., Batič, D.: The Effectiveness of Test-Driven Development: An Industrial Case Study. Software Quality Journal 19, 643–661 (2011)

    Article  Google Scholar 

  15. George, B., Williams, L.: An Initial Investigation of Test Driven Development in Industry. In: Proceedings of the 2003 ACM Symposium on Applied Computing, SAC 2003, pp. 1135–1139. ACM, New York (2003)

    Google Scholar 

  16. Geras, A., Smith, M., Miller, J.: A Prototype Empirical Evaluation of Test Driven Development. In: Proceedings of the 10th International Symposium on Software Metrics. METRICS 2004, pp. 405–416 (September 2004)

    Google Scholar 

  17. Maximilien, E., Williams, L.: Assessing Test-Driven Development at IBM. In: Proceedings of the 25th International Conference on Software Engineering, ICSE 2003, pp. 564–569 (May 2003)

    Google Scholar 

  18. Nagappan, N., Maximilien, E., Bhat, T., Williams, L.: Realizing Quality Improvement through Test Driven Development: Results and Experiences of Four Industrial Teams. Empirical Software Engineering 13, 289–302 (2008)

    Article  Google Scholar 

  19. Williams, L., Maximilien, E.M., Vouk, M.: Test-Driven Development as a Defect-Reduction Practice. In: Proceedings of the 14th International Symposium on Software Reliability Engineering, ISSRE 2003, pp. 34–45 (November 2003)

    Google Scholar 

  20. Janzen, D.S., Saiedian, H.: Does Test-Driven Development Really Improve Software Design Quality? IEEE Software 25(2), 77–84 (2008)

    Article  Google Scholar 

  21. Madeyski, L., Szała, Ł.: The Impact of Test-Driven Development on Software Development Productivity — An Empirical Study. In: Abrahamsson, P., Baddoo, N., Margaria, T., Messnarz, R. (eds.) EuroSPI 2007. LNCS, vol. 4764, pp. 200–211. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  22. Müller, M., Höfer, A.: The Effect of Experience on the Test-Driven Development Process. Empirical Software Engineering 12(6), 593–615 (2007)

    Article  Google Scholar 

  23. Desai, C., Janzen, D.S., Clements, J.: Implications of Integrating Test-Driven Development Into CS1/CS2 Curricula. In: Proceedings of the 40th ACM Technical Symposium on Computer Science Education, SIGCSE 2009, pp. 148–152. ACM, New York (2009)

    Google Scholar 

  24. Gupta, A., Jalote, P.: An Experimental Evaluation of the Effectiveness and Efficiency of the Test Driven Development. In: Proceedings of the First International Symposium on Empirical Software Engineering and Measurement, ESEM 2007, pp. 285–294 (September 2007)

    Google Scholar 

  25. Huang, L., Holcombe, M.: Empirical Investigation Towards the Effectiveness of Test First Programming. Information and Software Technology 51(1), 182–194 (2009)

    Article  Google Scholar 

  26. Janzen, D., Saiedian, H.: On the Influence of Test-Driven Development on Software Design. In: Proceedings of the 19th Conference on Software Engineering Education and Training, CSEET 2006, pp. 141–148 (April 2006)

    Google Scholar 

  27. Madeyski, L.: The Impact of Test-First Programming on Branch Coverage and Mutation Score Indicator of Unit Tests: An Experiment. Information and Software Technology 52(2), 169–184 (2010)

    Article  Google Scholar 

  28. Pančur, M., Ciglarič, M.: Impact of Test-Driven Development on Productivity, Code and Tests: A Controlled Experiment. Information and Software Technology 53(6), 557–573 (2011)

    Article  Google Scholar 

  29. Vu, J., Frojd, N., Shenkel-Therolf, C., Janzen, D.: Evaluating Test-Driven Development in an Industry-Sponsored Capstone Project. In: Proceedings of the Sixth International Conference on Information Technology, ITNG 2009, pp. 229–234. New Generations (April 2009)

    Google Scholar 

  30. Wilkerson, J., Nunamaker, J.J., Mercer, R.: Comparing the Defect Reduction Benefits of Code Inspection and Test-Driven Development. IEEE Transactions on Software Engineering 38(3), 547–560 (2012)

    Article  Google Scholar 

  31. Fenton, N.E., Pfleeger, S.L.: Software Metrics: A Rigorous and Practical Approach. PWS Publishing Company, Boston (1997)

    Google Scholar 

  32. Pezzé, M., Young, M.: Software Testing and Analysis: Process, Principles and Techniques. Wiley, Chichester (2008)

    MATH  Google Scholar 

  33. McCabe, T.: A Complexity Measure. IEEE Transactions on Software Engineering SE 2(4), 308–320 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  34. Chidamber, S., Kemerer, C.: A Metrics Suite for Object Oriented Design. IEEE Transactions on Software Engineering 20(6), 476–493 (1994)

    Article  Google Scholar 

  35. Cook, S., He, J., Harrison, R.: Dynamic and Static Views of Software Evolution. In: Proceedings of the IEEE International Conference on Software Maintenance, pp. 592–601 (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Helsinki, P.O. Box 68, Gustaf Hällströmin katu 2b, FI-00014, Finland

    Simo Mäkinen & Jürgen Münch

Authors
  1. Simo Mäkinen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jürgen Münch
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Software Technology and Interactive Systems, Vienna University of Technology, Vienna, Austria

    Dietmar Winkler

  2. Institute of Software Technology and Interactive Systems, Vienna University of Technology, Vienna, Austria

    Stefan Biffl

  3. Software Quality Lab GmbH, Linz, Austria

    Johannes Bergsmann

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Mäkinen, S., Münch, J. (2014). Effects of Test-Driven Development: A Comparative Analysis of Empirical Studies. In: Winkler, D., Biffl, S., Bergsmann, J. (eds) Software Quality. Model-Based Approaches for Advanced Software and Systems Engineering. SWQD 2014. Lecture Notes in Business Information Processing, vol 166. Springer, Cham. https://doi.org/10.1007/978-3-319-03602-1_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-03602-1_10

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-03601-4

  • Online ISBN: 978-3-319-03602-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics