Skip to main content
Log in

Case studies synthesis: a thematic, cross-case, and narrative synthesis worked example

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Case studies are largely used for investigating software engineering practices. They are characterized by their flexible nature, multiple forms of data collection, and are mostly informed by qualitative data. Synthesis of case studies is necessary to build a body of knowledge from individual cases. There are many methods for such synthesis, but they are yet not well explored in software engineering. The objective of this research is to demonstrate the similarities and differences of the results and conclusions when applying three different methods of synthesis, and to discuss the challenges of synthesizing evidence from reported case studies in SE. We describe a worked example of three such methods where three independent teams synthesized two studies that investigated critical factors of trust in outsourced projects through thematic synthesis and cross-case analysis, and compared these to each other and also to an already published narrative synthesis. In addition, despite that the primary studies were well presented for synthesis, we identified challenges in the use of case studies synthesis methods related to the goals and research questions of the synthesis, the types and number of case studies, variations in context, limited access to raw data, and quality of the case studies. Thus, we recommend that the analysts should be aware of these challenges and try to account for them during the execution of the synthesis. We also recommend that analysts consider using more than one method of synthesis for sake of reliability of the results and conclusions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Ashrafian H, Darsi A, Athanasiou T (2011) Evidence synthesis: evolving methodologies to optimise patient care and enhance policy decisions in evidence synthesis in healthcare. Springer-Verlag, London

    Google Scholar 

  • Babar MA, Verner JM, Nguyen PT (2007) Establishing and maintaining trust in software outsourcing relationships: an empirical investigation. J Syst Softw 80(9):1438–1449

    Article  Google Scholar 

  • Bethel EC, Bernard RM (2010) Developments and trends in synthesizing diverse forms of evidence: beyond comparisons between distance education and classroom instruction. Am J Dist Educ 31(3):231–256

    Article  Google Scholar 

  • Cruzes DS, Dybå T (2010) “Synthesizing evidence in software engineering research”, ESEM, 1–10

  • Cruzes DS, Dybå T (2011a) Research synthesis in software engineering: a tertiary study”. IST 53(5):440–455

    Google Scholar 

  • Cruzes DS, Dybå T (2011b) “Recommended Steps for Thematic Synthesis in Software Engineering”, ESEM: 275–284

  • Cruzes DS, Dybå T, Runeson P, Höst M (2011) Case Studies Synthesis: Brief Experience and Challenges for the Future. ESEM: 343–346

  • Dybå T (2013) Contextualizing empirical evidence. IEEE Softw 30(1):81–83

    Article  Google Scholar 

  • Dybå T, Dingsøyr T (2008) Strength of Evidence in Systematic Reviews in Software Engineering, ESEM’, pp 178–187.

  • Dybå T, Sjøberg DIK, Cruzes DS (2012) What works for whom, where, when, and why?: on the role of context in empirical software engineering. ESEM: 19–28

  • Gerring J (2007) Case study research: principles and practices. Cambridge Univ. Press, New York

    Google Scholar 

  • Khan S, VanWynsberghe R (2008) Cultivating the Under-Mined: Cross-Case Analysis as Knowledge Mobilization. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 9 (1), Art. 34, http://nbn-resolving.de/urn:nbn:de:0114-fqs0801348

  • Kitchenham BA, Dybå T, Jørgensen M (2004) Evidence-based Software Engineering, Proc. ICSE’04, Edinburgh, Scotland, 23–28 May, pp. 273–281

  • Kitchenham BA, Charters S (2007) Guidelines for performing Systematic Literature Reviews in Software Engineering, Version 2.3, Keele University, EBSE Technical Report, EBSE-2007-01

  • Kitchenham BA, Sjøberg DIK, Dybå T, Pfahl D, Brereton P, Budgen D, Höst M, Runeson P (2012) Three empirical studies on the agreement of reviewers about the quality of software engineering experiments. Inf Softw Technol 54(8):804–819

    Article  Google Scholar 

  • Larsson R (1993) Case survey methodology: quantitative analysis of patterns across case studies. Acad Manag J 36(6):1515–1546

    Article  MathSciNet  Google Scholar 

  • Lucas WA (1974): The case survey method: Aggregating case experience. Santa Monica

  • Mays N, Pope C, Popay J (2005) Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy 10:6–20

    Article  Google Scholar 

  • Miles MB, Huberman AM (1984) Qualitative Data Analysis: A Sourcebook of New Methods, Sage

  • Miles MB, Huberman AM (1994) Qualitative Data Analysis: An Expanded Source Book, Sage

  • Newig J, Fritsch O (2009) The case survey method and applications in political science. APSA 2009 Paper. Available at SSRN: ssrn.com/abstract = 1451643 (Toronto)

  • Nguyen PT, Babar MA, Verner JM (2006) Critical factors in establishing and maintaining trust in software outsourcing relationships. EASE, http://ewic.bcs.org/content/ConWebDoc/3709

  • Oza N, Hall T, Rainer A, Grey S (2005) Trust in software outsourcing relationships: an empirical investigation of Indian software companies. In: 9th International Conference on Empirical Assessment in Software Engineering (EASE 2005). Keele, UK

  • Oza N, Hall T, Rainer A, Grey S (2006) Trust in software outsourcing relationships: an empirical investigation of Indian software companies. IST 48:345–354

    Google Scholar 

  • Popay J, Roberts H, Sowden A, Petticrew M, Britten N, Arai L, Roen K, Rodgers M (2010) Developing methods for the narrative synthesis of quantitative and qualitative data in systematic reviews of effects, Centre for Review and Dissemination, ESRC, http://www.york.ac.uk/inst/crd/projects/narrative_synthesis.htm. Assessed in Dec. 2012

  • Pope C, Mays N, Popay J (2007) Synthesizing Qualitative and Quantitative Health Evidence: A Guide to Methods, Open University Press

  • Ragin CC (1987) The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies, Univ. of California Press

  • Runeson P, Höst M (2009) Guidelines for conducting and reporting case study research in software engineering”. Empir Softw Eng 14(2):131–164

    Article  Google Scholar 

  • Runeson P, Höst M, Rainer A, Regnell B (2012) Case Study Research in Software Engineering – Guidelines and Examples. Wiley, first edition, ISBN-10: 1118104358, ISBN-13: 978–1118104354

  • Šmite D, Wohlin C, Gorschek T, Feldt R (2010) Empirical evidence in global software engineering: a systematic review”. Empir Softw Eng 15(1):91–118

    Article  Google Scholar 

  • Thomas J, Harden A (2008) Methods for the thematic synthesis of qualitative research in systematic reviews”. BMC Med Res Methodol 8:45

    Article  MATH  Google Scholar 

  • Yin RK (2014) Case study research. Design and methods, 5th edn. Sage, London

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Daniela S. Cruzes.

Additional information

Communicated by: Maurizio Morisio

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cruzes, D.S., Dybå, T., Runeson, P. et al. Case studies synthesis: a thematic, cross-case, and narrative synthesis worked example. Empir Software Eng 20, 1634–1665 (2015). https://doi.org/10.1007/s10664-014-9326-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-014-9326-8

Keywords

Navigation