Sandro Schulze
Jörg Liebig
Sven Apel
ABSTRACT
The C preprocessor (CPP) is a simple and language-independent tool, widely used to implement variable software systems using conditional compilation (i.e., by including or excluding annotated code). Although CPP provides powerful means to express variability, it has been criticized for allowing arbitrary annotations that break the underlying structure of the source code. We distinguish between disciplined annotations, which align with the structure of the source code, and undisciplined annotations, which do not. Several studies suggest that especially the latter type of annotations makes it hard to (automatically) analyze the code. However, little is known about whether the type of annotations has an effect on program comprehension. We address this issue by means of a controlled experiment with human subjects. We designed similar tasks for both, disciplined and undisciplined annotations, to measure program comprehension. Then, we measured the performance of the subjects regarding correctness and response time for solving the tasks. Our results suggest that there are no differences between disciplined and undisciplined annotations from a program-comprehension perspective. Nevertheless, we observed that finding and correcting errors is a time-consuming and tedious task in the presence of preprocessor annotations.
- T. Anderson and J. Finn. The New Statistical Analysis of Data. Springer, 1996.Google Scholar
Cross Ref
- B. Baker. On Finding Duplication and Near-Duplication in Large Software Systems. In Proc. Work. Conf. Reverse Engineering (WCRE), pages 86--95. IEEE, 1995. Google ScholarDigital Library
- I. Baxter and M. Mehlich. Preprocessor Conditional Removal by Simple Partial Evaluation. In Proc. Work. Conf. Reverse Engineering (WCRE), pages 281--290. IEEE, 2001. Google ScholarDigital Library
- G. Camilli and K. D. Hopkins. Applicability of Chi-square to 2 x 2 Contingency Tables with Small Expected Cell Frequencies. Psychological Bulletin, 85(1):163, 1978.Google ScholarCross Ref
- A. Dunsmore and M. Roper. A Comparative Evaluation of Program Comprehension Measures. Journal Sys. and Soft. (JSS), 52(3):121--129, 2000.Google Scholar
- M. Ernst, G. Badros, and D. Notkin. An Empirical Analysis of C Preprocessor Use. IEEE Trans. Software Engineering (TSE), 28(12):1146--1170, 2002. Google ScholarDigital Library
- J.-M. Favre. The CPP Paradox. In Proc. European Workshop Software Maintenance, 1995. http://equipes-lig.imag.fr/adele/Les.Publications/intConferences/EWSM91995Fav.pdf.Google Scholar
- J.-M. Favre. Understanding-In-The-Large. In Int. Workshop Program Comprehension (IWPC), pages 29--38. IEEE, 1997. Google ScholarDigital Library
- J. Feigenspan, C. Kästner, S. Apel, J. Liebig, M. Schulze, R. Dachselt, M. Papendieck, T. Leich, and G. Saake. Do background colors improve program comprehension in the #ifdef hell? Empirical Software Engineering, pages 1--47, 2012. Google ScholarDigital Library
- J. Feigenspan, C. Kästner, J. Liebig, S. Apel, and S. Hanenberg. Measuring Programming Experience. In Proc. Int. Conf. Program Comprehension (ICPC), pages 73--82. IEEE, 2012.Google ScholarCross Ref
- J. Feigenspan, M. Schulze, M. Papendieck, C. Kästner, R. Dachselt, V. Köppen, and M. Frisch. Using Background Colors to Support Program Comprehension in Software Product Lines. In Proc. Int. Conf. Evaluation and Assessment in Software Engineering (EASE), pages 66--75. Institution of Engineering and Technology, 2011.Google ScholarCross Ref
- J. Feigenspan, M. Schulze, M. Papendieck, C. Kästner, R. Dachselt, V. Köppen, M. Frisch, and G. Saake. Supporting Program Comprehension in Large Preprocessor-Based Software Product Lines. IET Software, 6(6):488--501, 2012.Google ScholarCross Ref
- J. Feigenspan, N. Siegmund, A. Hasselberg, and M. Köppen. PROPHET: Tool Infrastructure to Support Program Comprehension Experiments. In Proc. Int. Symp. Empirical Software Engineering and Measurement (ESEM), 2011. Poster.Google Scholar
- A. Garrido and R. Johnson. Challenges of Refactoring C Programs. In Proc. Int. Workshop Principles of Software Evolution (IWPSE), pages 6--14. ACM, 2002. Google ScholarDigital Library
- A. Garrido and R. Johnson. Refactoring C with Conditional Compilation. In Proc. Int. Conf. Automated Software Engineering (ASE), pages 323--326. IEEE, 2003.Google ScholarDigital Library
- A. Garrido and R. Johnson. Analyzing Multiple Configurations of a C Program. In Proc. Int. Conf. Software Maintenance (ICSM), pages 379--388. IEEE, 2005. Google ScholarDigital Library
- N. Göde and J. Harder. Clone Stability. In Proc. European Conf. Software Maintenance and Reengineering (CSMR), pages 65--74. IEEE, 2011. Google ScholarDigital Library
- C. Goodwin. Research in Psychology: Methods and Design. Wiley Publishing, Inc., second edition, 1999.Google Scholar
- A. Jedlitschka, M. Ciolkowski, and D. Pfahl. Reporting Experiments in Software Engineering. In Guide to Advanced Empirical Software Engineering, pages 201--228. Springer, 2008.Google ScholarCross Ref
- E. Jürgens, F. Deissenböck, B. Hummel, and S. Wagner. Do Code Clones Matter? In Proc. Int. Conf. Software Engineering (ICSE), pages 485--495. IEEE, 2009. Google ScholarDigital Library
- C. Kapser and M. W. Godfrey. "Cloning Considered Harmful" Considered Harmful. In Proc. Work. Conf. Reverse Engineering (WCRE), pages 19--28. IEEE, 2006. Google ScholarDigital Library
- C. Kästner, S. Apel, and M. Kuhlemann. Granularity in Software Product Lines. In Proc. Int. Conf. Software Engineering (ICSE), pages 311--320. ACM, 2008. Google ScholarDigital Library
- C. Kästner, S. Apel, S. Trujillo, M. Kuhlemann, and D. Batory. Guaranteeing Syntactic Correctness for all Product Line Variants: A Language-Independent Approach. In Proc. Int. Conf. Objects, Models, Components, Patterns (TOOLS), pages 174--194. Springer, 2009.Google ScholarCross Ref
- T. D. LaToza, G. Venolia, and R. DeLine. Maintaining Mental Models: A Study of Developer Work Habits. In Proc. Int. Conf. Software Engineering (ICSE), pages 492--501. ACM, 2006. Google ScholarDigital Library
- D. Le, E. Walkingshaw, and M. Erwig. #ifdef Confirmed Harmful: Promoting Understandable Software Variation. In Proc. IEEE Symp. Visual Languages and Human-Centric Computing (VL/HCC), pages 143--150. IEEE, 2011.Google Scholar
- J. Liebig, S. Apel, C. Lengauer, C. Kästner, and M. Schulze. An Analysis of the Variability in Forty Preprocessor-Based Software Product Lines. In Proc. Int. Conf. Software Engineering (ICSE), pages 105--114. ACM, 2010. Google ScholarDigital Library
- J. Liebig, C. Kästner, and S. Apel. Analyzing the Discipline of Preprocessor Annotations in 30 Million Lines of C Code. In Proc. Int. Conf. Aspect-Oriented Software Development (AOSD), pages 191--202. ACM, 2011. Google ScholarDigital Library
- R. Likert. A Technique for the Measurement of Attitudes. Archives of Psychology, 140:1--55, 1932.Google Scholar
- F. Medeiros, M. Ribeiro, and R. Gheyi. Investigating Preprocessor-Based Syntax Errors. In Proc. Int. Conf. Generative Programming and Component Engineering (GPCE). ACM, 2013. to appear. Google ScholarDigital Library
- C. Roy and J. Cordy. A Survey on Software Clone Detection Research. Technical Report 2007-541, Queen's University at Kingston, 2007.Google Scholar
- S. Schulze, E. Jürgens, and J. Feigenspan. Analyzing the Effect of Preprocessor Annotations on Code Clones. In Proc. Work. Conf. Source Code Analysis and Manipulation (SCAM), pages 115--124. IEEE, 2011. Google ScholarDigital Library
- W. R. Shadish, T. D. Cook, and D. T. Campbell. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company, 2002.Google Scholar
- J. Siegmund. Framework for Measuring Program Comprehension. PhD thesis, University of Magdeburg, 2012.Google Scholar
- H. Spencer and G. Collyer. #ifdef Considered Harmful, or Portability Experience with C News. In Proc. USENIX Technical Conf., pages 185--197. USENIX Association Berkeley, 1992.Google Scholar
- M. Svahnberg, A. Aurum, and C. Wohlin. Using Students as Subjects - An Empirical Evaluation. In Proc. Int. Symp. Empirical Software Engineering and Measurement (ESEM), pages 288--290. ACM, 2008. Google ScholarDigital Library
- J. Yellott. Correction for Fast Guessing and the Speed Accuracy Trade-off in Choice Reaction Time. Journal of Mathematical Psychology, 8:159--199, 1971.Google ScholarCross Ref
Index Terms
- Does the discipline of preprocessor annotations matter?: a controlled experiment
Recommendations
Does the discipline of preprocessor annotations matter?: a controlled experiment
GPCE '13The C preprocessor (CPP) is a simple and language-independent tool, widely used to implement variable software systems using conditional compilation (i.e., by including or excluding annotated code). Although CPP provides powerful means to express ...
Analyzing the discipline of preprocessor annotations in 30 million lines of C code
AOSD '11: Proceedings of the tenth international conference on Aspect-oriented software developmentThe C preprocessor cpp is a widely used tool for implementing variable software. It enables programmers to express variable code (which may even crosscut the entire implementation) with conditional compilation. The C preprocessor relies on simple text ...
Analyzing the Effect of Preprocessor Annotations on Code Clones
SCAM '11: Proceedings of the 2011 IEEE 11th International Working Conference on Source Code Analysis and ManipulationThe C preprocessor cpp is a powerful and language-independent tool, widely used to implement variable software in different programming languages (C, C++) using conditional compilation. Preprocessor annotations can used on different levels of ...
Comments