Skip to main content
Log in

A Controlled Experiment Comparing the Maintainability of Programs Designed with and without Design Patterns—A Replication in a Real Programming Environment

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Software “design patterns” seek to package proven solutions to design problems in a form that makes it possible to find, adapt and reuse them. To support the industrial use of design patterns, this research investigates when, and how, using patterns is beneficial, and whether some patterns are more difficult to use than others. This paper describes a replication of an earlier controlled experiment on design patterns in maintenance, with major extensions. Experimental realism was increased by using a real programming environment instead of pen and paper, and paid professionals from multiple major consultancy companies as subjects. Measurements of elapsed time and correctness were analyzed using regression models and an estimation method that took into account the correlations present in the raw data. Together with on-line logging of the subjects’ work, this made possible a better qualitative understanding of the results. The results indicate quite strongly that some patterns are much easier to understand and use than others. In particular, the Visitor pattern caused much confusion. Conversely, the patterns Observer and, to a certain extent, Decorator were grasped and used intuitively, even by subjects with little or no knowledge of patterns. The implication is that design patterns are not universally good or bad, but must be used in a way that matches the problem and the people. When approaching a program with documented design patterns, even basic training can improve both the speed and quality of maintenance activities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alexander, C. 1978. A Pattern Language: Towns, Buildings, Construction. New York, USA: Oxford University Press, Inc.

    Google Scholar 

  • Alexander, C. 1987. The Timeless Way of Building. New York, USA: Oxford University Press Inc.

    Google Scholar 

  • Arisholm, E. 2001. Empirical assessment of changeability in object-oriented software. Ph.D. thesis, University of Oslo.

  • Arisholm, E., Sjøberg, D. I. K., Carelius, G. J., and Lindsjørn, Y. 2002a. SESE an experiment support environment for evaluating software engineering technologies. In NW-PER2002 (Tenth Nordic Workshop on Programming and Software Development Tools and Techniques). Copenhagen, Denmark, pp. 81-98.

  • Arisholm, E., Sjøberg, D. I. K., Carelius, G. J., and Lindsjørn, Y. 2002b. A web-based support environment for software engineering experiments. Nordic Journal of Computing 9(4): 231–247.

    Google Scholar 

  • Arisholm, E., Sjøberg, D. I. K., and Jørgensen, M. 2001. Assessing the changeability of two objectoriented design alternatives-a controlled experiment. Empirical Software Engineering 6, 231–277.

    Google Scholar 

  • Buschmann, F., Meunier, R., Rohnert, H., Sommerlad, P., and Stal, M. 1996. Pattern-Oriented Software Architecture. Chichester: Wiley.

    Google Scholar 

  • Diggle, P., Liang, K., and Zeger, S. 1994. The Analysis of Longitudinal Data. Oxford: Oxford University Press.

    Google Scholar 

  • Efron, B., and Tibshirani, R. J. 1993. An Introduction to the Bootstrap. Monographs on Statistics and Applied Probability. London: Chapman &; Hall.

    Google Scholar 

  • Gamma, E., Helm, R., Johnson, R., and Vlissides, J. 1995. Design Patterns: Elements of Reusable Object-Oriented Software. Reading, MA: Addison-Wesley.

    Google Scholar 

  • Kitchenham, B. A., Pfleeger, S. L., Pickard, L. M., Jones, P. W., Hoaglin, D. C., El-Emam, K., and Rosenberg, J. 2002. Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering 28(8): 721–734.

    Google Scholar 

  • Liang, K., and Zeger, S. 1986. Longitudinal Data Analysis using Generalized Linear Models. Biometrika 73, 13–22.

    Google Scholar 

  • Lindsay, R., and Ehrenberg, A. 1993. The design of replicated studies. The American Statistician 47(3): 217–228.

    Google Scholar 

  • McCullagh, P., and Nelder, J. 1989. Generalized Linear Models. New York: Chapman and Hall.

    Google Scholar 

  • Prechelt, L. 2000. An empirical study of working speed differences between software engineers for various kinds of task. Submitted to IEEE Transactions on Software Engineering, to be revised.

  • Prechelt, L., Unger, B., Tichy, W. F., Brössler, P., and Votta, L. G. 2001. A controlled experiment in maintenance comparing design patterns to simpler solutions. IEEE Transactions on Software Engineering 27(12): 1134–1144.

    Google Scholar 

  • Sjøberg, D., Anda, B., Arisholm, E., Dybå, T., Jørgensen, M., Karahasanovic, A., Koren, E., and Vokác, M. 2002. Conducting realistic experiments in software engineering. In ISESE2002 (First International Symposium on Empirical Software Engineering). Nara, Japan, pp. 17-26, IEEE Computer Society.

  • Smith, D., Robertson, W., and Diggle, P. 1996. Object-oriented software for the analysis of longitudinal data in S. Technical Report Technical Report MA96/192, Department of Mathematics and Statistics, University of Lancaster.

  • Yourdon, E. 1999. Death March. Indianapolis, Indiana: Prentice Hall PTR.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vokáč, M., Tichy, W., SjØberg, D.I.K. et al. A Controlled Experiment Comparing the Maintainability of Programs Designed with and without Design Patterns—A Replication in a Real Programming Environment. Empirical Software Engineering 9, 149–195 (2004). https://doi.org/10.1023/B:EMSE.0000027778.69251.1f

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:EMSE.0000027778.69251.1f

Navigation