Abstract
Software “design patterns” seek to package proven solutions to design problems in a form that makes it possible to find, adapt and reuse them. To support the industrial use of design patterns, this research investigates when, and how, using patterns is beneficial, and whether some patterns are more difficult to use than others. This paper describes a replication of an earlier controlled experiment on design patterns in maintenance, with major extensions. Experimental realism was increased by using a real programming environment instead of pen and paper, and paid professionals from multiple major consultancy companies as subjects. Measurements of elapsed time and correctness were analyzed using regression models and an estimation method that took into account the correlations present in the raw data. Together with on-line logging of the subjects’ work, this made possible a better qualitative understanding of the results. The results indicate quite strongly that some patterns are much easier to understand and use than others. In particular, the Visitor pattern caused much confusion. Conversely, the patterns Observer and, to a certain extent, Decorator were grasped and used intuitively, even by subjects with little or no knowledge of patterns. The implication is that design patterns are not universally good or bad, but must be used in a way that matches the problem and the people. When approaching a program with documented design patterns, even basic training can improve both the speed and quality of maintenance activities.
Similar content being viewed by others
References
Alexander, C. 1978. A Pattern Language: Towns, Buildings, Construction. New York, USA: Oxford University Press, Inc.
Alexander, C. 1987. The Timeless Way of Building. New York, USA: Oxford University Press Inc.
Arisholm, E. 2001. Empirical assessment of changeability in object-oriented software. Ph.D. thesis, University of Oslo.
Arisholm, E., Sjøberg, D. I. K., Carelius, G. J., and Lindsjørn, Y. 2002a. SESE an experiment support environment for evaluating software engineering technologies. In NW-PER2002 (Tenth Nordic Workshop on Programming and Software Development Tools and Techniques). Copenhagen, Denmark, pp. 81-98.
Arisholm, E., Sjøberg, D. I. K., Carelius, G. J., and Lindsjørn, Y. 2002b. A web-based support environment for software engineering experiments. Nordic Journal of Computing 9(4): 231–247.
Arisholm, E., Sjøberg, D. I. K., and Jørgensen, M. 2001. Assessing the changeability of two objectoriented design alternatives-a controlled experiment. Empirical Software Engineering 6, 231–277.
Buschmann, F., Meunier, R., Rohnert, H., Sommerlad, P., and Stal, M. 1996. Pattern-Oriented Software Architecture. Chichester: Wiley.
Diggle, P., Liang, K., and Zeger, S. 1994. The Analysis of Longitudinal Data. Oxford: Oxford University Press.
Efron, B., and Tibshirani, R. J. 1993. An Introduction to the Bootstrap. Monographs on Statistics and Applied Probability. London: Chapman &; Hall.
Gamma, E., Helm, R., Johnson, R., and Vlissides, J. 1995. Design Patterns: Elements of Reusable Object-Oriented Software. Reading, MA: Addison-Wesley.
Kitchenham, B. A., Pfleeger, S. L., Pickard, L. M., Jones, P. W., Hoaglin, D. C., El-Emam, K., and Rosenberg, J. 2002. Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering 28(8): 721–734.
Liang, K., and Zeger, S. 1986. Longitudinal Data Analysis using Generalized Linear Models. Biometrika 73, 13–22.
Lindsay, R., and Ehrenberg, A. 1993. The design of replicated studies. The American Statistician 47(3): 217–228.
McCullagh, P., and Nelder, J. 1989. Generalized Linear Models. New York: Chapman and Hall.
Prechelt, L. 2000. An empirical study of working speed differences between software engineers for various kinds of task. Submitted to IEEE Transactions on Software Engineering, to be revised.
Prechelt, L., Unger, B., Tichy, W. F., Brössler, P., and Votta, L. G. 2001. A controlled experiment in maintenance comparing design patterns to simpler solutions. IEEE Transactions on Software Engineering 27(12): 1134–1144.
Sjøberg, D., Anda, B., Arisholm, E., Dybå, T., Jørgensen, M., Karahasanovic, A., Koren, E., and Vokác, M. 2002. Conducting realistic experiments in software engineering. In ISESE2002 (First International Symposium on Empirical Software Engineering). Nara, Japan, pp. 17-26, IEEE Computer Society.
Smith, D., Robertson, W., and Diggle, P. 1996. Object-oriented software for the analysis of longitudinal data in S. Technical Report Technical Report MA96/192, Department of Mathematics and Statistics, University of Lancaster.
Yourdon, E. 1999. Death March. Indianapolis, Indiana: Prentice Hall PTR.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Vokáč, M., Tichy, W., SjØberg, D.I.K. et al. A Controlled Experiment Comparing the Maintainability of Programs Designed with and without Design Patterns—A Replication in a Real Programming Environment. Empirical Software Engineering 9, 149–195 (2004). https://doi.org/10.1023/B:EMSE.0000027778.69251.1f
Issue Date:
DOI: https://doi.org/10.1023/B:EMSE.0000027778.69251.1f