Minireviews Open Access
Copyright ©The Author(s) 2016. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Methodol. Mar 26, 2016; 6(1): 101-104
Published online Mar 26, 2016. doi: 10.5662/wjm.v6.i1.101
Study design in evidence-based surgery: What is the role of case-control studies?
Amy M Cao, Michael R Cox, Guy D Eslick, Discipline of Surgery, the University of Sydney, Nepean Hospital, Penrith, NSW 2751, Australia
Author contributions: All authors contributed to this manuscript.
Conflict-of-interest statement: There are no conflicts of interest arising from this work.
Open-Access: This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Correspondence to: Guy D Eslick, Professor, Discipline of Surgery, the University of Sydney, Nepean Hospital, Level 3, Clinical Building, P.O. Box 63, Penrith, NSW 2751, Australia. guy.eslick@sydney.edu.au
Telephone: +61-2-47341373 Fax: +61-2-47343432
Received: August 6, 2015
Peer-review started: August 10, 2015
First decision: November 3, 2015
Revised: December 17, 2015
Accepted: December 29, 2015
Article in press: January 4, 2016
Published online: March 26, 2016

Abstract

Randomized controlled trials (RCTs) are the gold standard in terms of study design, however, in the surgical setting conducting RCTs can often be unethical or logistically impossible. Case-control studies should become the major study design used in surgical research when RCTs are unable to be conducted and definitely replacing case series which offer little insight into surgical outcomes and disease processes.

Key Words: Research studies, Case-control studies, Randomized clinical trials, Bias, Sample size

Core tip: Case-control studies should be utilized more often in the surgical setting for research purposes. They offer many advantages to other study designs, especially when the option of conducting a randomized clinical trial may be impractical or not ethically feasible.



INTRODUCTION

The hierarchy of study design is well ingrained in determining the quality and subsequent acceptance of clinical evidence (Figure 1). Randomised controlled trials (RCT) are considered the gold standard study design and the “most scientifically rigorous method for hypothesis testing”, with results from many non-randomised trials prejudiced by doubts of study reliability, bias and accuracy[1-3]. Yet in certain aspects of surgery, RCTs may be difficult to conduct and indeed the number of surgical RCTs is known to be limited in comparison[4].

Figure 1
Figure 1 Hierarchy of study design.

RCTs involve the comparison of outcomes after random allocation of a particular intervention to a patient group with a control group whilst case-control studies (CCS) involve observing outcome differences between patients with a particular disease (cases) and those without the disease (control). It is commonly accepted that results from RCTs provide superior evidence in the evaluation of a therapeutic intervention when compared to CCS. However, there are many considerations that result in flaws in this concept especially in surgery. Difficulties in standardising surgical technique, variable learning curves in introducing new or modifications of an operation and difficulties in recruiting patients leading to underpowered studies need to be recognised[5]. In fact results from poorly designed RCTs can have the undue advantage of being perceived and accepted as the “superior study design” with more robust findings[6]. The aim of this paper is to explore various factors influencing the role of CCS in the surgical context and provide recommendations to improving the quality of CCS.

POWER

The strength of CCS lie in its ability to recruit larger sample sizes, resultant increase in the power of studies, lower cost and the ability to be conducted in “greater timeliness” (Table 1)[7]. CCS also have the ability to report rare infrequent adverse effects, e.g., bile duct injuries in laparoscopic cholecystectomies[8,9]. As CCS may be performed by researchers with limited resources, larger patient populations are able to be recruited compared with RCTs which generally require more expert support from epidemiologists and require financial support[10]. Lack of funding and resource constraints have been cited as major obstacles in conducting RCTs[11]. Inadequate sample sizes lead to underpowered RCTs which may miss clinically important benefits and lead to type II error[12]. Type II error is the failure to reject the null hypothesis when it is false, i.e., False negative results[12].

Table 1 Advantages and disadvantages of case control studies.
Advantages of case control studiesDisadvantages of case control studies
Ability to investigate low incidence outcomesRisk of bias
Ability to recruit large sample sizeConfounding factors
Relative ease and efficiencyRequires careful selection of controls
May be conducted in shorter time frameWeaker evidence of causality (20)
Relatively low costBlinding is not possible
CLINICAL APPLICABILITY

A particular strength of CCS is the inclusion of data from practical clinical scenarios. RCTs, whilst limiting potential confounding variables, provide evidence from data collected from highly rigid experimental models[13]. In investigating certain surgical techniques such as laparoscopic cholecystectomy, strict criteria such as those excluding obese patients and patients with multiple comorbidities are likely to lead to results inconsistent with the clinical setting and limit the practicality of findings. In patients who have rare or life threatening illnesses, it will be difficult to include them in RCT[6]. In addition unlike CCS, RCT also tend to limit the spectrum of disease represented compared to observational studies[2,6].

RANDOMISATION AND CONFOUNDERS

Non-randomised observational studies such as CCS and cohort studies are more prone to bias than RCT due to lack of randomisation. The randomisation process aims to minimise systematic error and eliminate or at least equilibrate confounding factors between both treatment and control groups. It is more difficult for observational studies to allow for this equilibration and hence is more prone to bias. Without randomisation, it may be unclear why certain patients were assigned to a particular intervention whilst others were not[9]. However, whilst randomisation can limit bias, it may not be feasible or ethical in the surgical context. For example, it may be unethical to deny one group of patients the treatment benefits of well established “gold standard” interventions[14]. In addition, it may be difficult to recruit patients who will leave their choice of treatment up to chance alone and accept the process of randomisation[11].

Whilst it is more difficult for CCS to account for confounding factors, it is not impossible without randomisation. Matching controls with cases is one potential method[15]. Matching where controls are specifically selected for their similarity to the treatment group in particular characteristics such as age, sex, socioeconomic status, body mass index, etc., can be used to equilibrate potential confounders in CCS.

Allocation concealment and blinding

Furthermore in surgery, allocation concealment and blinding may be impractical and unethical. In most major surgical procedures, it would be unethical to expose patients in the control groups to the risks of sham operations. Whilst various techniques have been used in the blinding of patients in surgery including the use of multiple wound dressings over intact skin, the efficacy of such blinding techniques is unclear.

Bias

The concern that observational studies can bias evidence by finding stronger treatment associations than RCTs has been reported in the literature[7,16]. However comparisons between results for observational and RCTs in other studies have shown results to be similar between the two in most outcomes[7]. For example one study analysed the results of meta-analyses comparing RCTs and well-designed observational studies (cohort and case control studies) on a range of treatments including hypertension treatment and CHD, Bacillus Calmette-Guerin vaccine in tuberculosis, mammography screening for breast cancer and found results from observational studies “did not systematically overestimate the magnitude of exposure-outcome associations reported in RCTs”[2,7]. An explanation for the noted differences in some studies between RCT and CCS potentially results from less robustly designed CCS were used to generate generalised conclusions regarding observational studies[2].

Recommendations to improve CCS

It would be imprudent to argue that CCS provide a superior level of evidence to RCT. However, CCS can often provide additional and more clinically relevant evidence that can complement data derived from RCTs. There are various means of ensuring high quality CCS. Recommendations to ensuring sound CCS evidence include: (1) encourage use of STROBE statement to ensure adequate reporting of outcomes[7]; (2) develop an exhaustive database of baseline characteristics and variables during data collection stage of CCS; (3) design CCS to test the clinical applicability and generalizability of results from RCT rather than formulating hypothesis to investigate[17-19]; (4) ensure adequate statistical power and sample size by performing sample size and power calculations prior to the initiation of studies; (5) appropriate statistical techniques for the clinical question, e.g., Propensity analysis to match patients, use of risk adjusted statistical models; and (6) encourage sound methodology techniques such as intention to treat and adequate follow-up.

CONCLUSION

Well-designed RCTs undoubtedly provide powerful estimates of treatment effects. However, they are time-consuming, costly, difficult to conduct especially in surgery and can be misinterpreted when data is extrapolated outside the experiment sample. CCS on the other hand have the ability to recruit large sample sizes, are more efficient to conduct and allow for the examination of variables in the clinical setting. It is unfortunate that CCS are often undervalued and under-utilised in surgery. RCT and CCS provide evidence that is complementary to each other. Greater understanding is required in appraising RCT and CCS in the surgical environment.

Footnotes

P- Reviewer: Ni Y S- Editor: Qiu S L- Editor: A E- Editor: Liu SQ

References
1.  DiPietro NA. Methods in epidemiology: observational study designs. Pharmacotherapy. 2010;30:973-984.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 51]  [Cited by in F6Publishing: 54]  [Article Influence: 4.2]  [Reference Citation Analysis (0)]
2.  Concato J. Observational versus experimental studies: what’s the evidence for a hierarchy? NeuroRx. 2004;1:341-347.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 111]  [Cited by in F6Publishing: 116]  [Article Influence: 6.1]  [Reference Citation Analysis (0)]
3.  Hillner BE, Mandelblatt J. Caring for older women with breast cancer: can observational research fill the clinical trial gap? J Natl Cancer Inst. 2006;98:660-661.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 25]  [Cited by in F6Publishing: 27]  [Article Influence: 1.5]  [Reference Citation Analysis (0)]
4.  Ahmed Ali U, van der Sluis PC, Issa Y, Habaga IA, Gooszen HG, Flum DR, Algra A, Besselink MG. Trends in worldwide volume and methodological quality of surgical randomized controlled trials. Ann Surg. 2013;258:199-207.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 41]  [Cited by in F6Publishing: 42]  [Article Influence: 3.8]  [Reference Citation Analysis (0)]
5.  Curry JI, Reeves B, Stringer MD. Randomized controlled trials in pediatric surgery: could we do better? J Pediatr Surg. 2003;38:556-559.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 41]  [Cited by in F6Publishing: 41]  [Article Influence: 2.0]  [Reference Citation Analysis (0)]
6.  McCulloch P, Taylor I, Sasako M, Lovett B, Griffin D. Randomised trials in surgery: problems and possible solutions. BMJ. 2002;324:1448-1451.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 514]  [Cited by in F6Publishing: 524]  [Article Influence: 23.8]  [Reference Citation Analysis (0)]
7.  Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. N Engl J Med. 2000;342:1878-1886.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1576]  [Cited by in F6Publishing: 1468]  [Article Influence: 61.2]  [Reference Citation Analysis (0)]
8.  Cao AM, Eslick GD, Cox MR. Early laparoscopic cholecystectomy is superior to delayed acute cholecystitis: a meta-analysis of case-control studies. Surg Endosc. 2016;30:1172-1182.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 2]  [Cited by in F6Publishing: 2]  [Article Influence: 0.2]  [Reference Citation Analysis (0)]
9.  MacMahon S, Collins R. Reliable assessment of the effects of treatment on mortality and major morbidity, II: observational studies. Lancet. 2001;357:455-462.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 167]  [Cited by in F6Publishing: 173]  [Article Influence: 7.5]  [Reference Citation Analysis (0)]
10.  Comber H, Perry IJ. Observational studies for intervention assessment. Lancet. 2001;357:2141-2142.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 6]  [Cited by in F6Publishing: 5]  [Article Influence: 0.2]  [Reference Citation Analysis (0)]
11.  Gattellari M, Ward JE, Solomon MJ. Randomized, controlled trials in surgery: perceived barriers and attitudes of Australian colorectal surgeons. Dis Colon Rectum. 2001;44:1413-1420.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 31]  [Cited by in F6Publishing: 31]  [Article Influence: 1.3]  [Reference Citation Analysis (0)]
12.  Maggard MA, O’Connell JB, Liu JH, Etzioni DA, Ko CY. Sample size calculations in surgery: are they done correctly? Surgery. 2003;134:275-279.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 42]  [Cited by in F6Publishing: 44]  [Article Influence: 2.1]  [Reference Citation Analysis (0)]
13.  Jahn A, Razum O. Observational studies for intervention assessment. Lancet. 2001;357:2141.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 8]  [Cited by in F6Publishing: 9]  [Article Influence: 0.4]  [Reference Citation Analysis (0)]
14.  McLeod RS. Issues in surgical randomized controlled trials. World J Surg. 1999;23:1210-1214.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 132]  [Cited by in F6Publishing: 133]  [Article Influence: 5.3]  [Reference Citation Analysis (0)]
15.  de Graaf MA, Jager KJ, Zoccali C, Dekker FW. Matching, an appealing method to avoid confounding? Nephron Clin Pract. 2011;118:c315-c318.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 46]  [Cited by in F6Publishing: 49]  [Article Influence: 3.8]  [Reference Citation Analysis (0)]
16.  Chalmers TC, Celano P, Sacks HS, Smith H. Bias in treatment assignment in controlled clinical trials. N Engl J Med. 1983;309:1358-1361.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 644]  [Cited by in F6Publishing: 663]  [Article Influence: 16.2]  [Reference Citation Analysis (0)]
17.  von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61:344-349.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 5754]  [Cited by in F6Publishing: 7009]  [Article Influence: 438.1]  [Reference Citation Analysis (0)]
18.  Hannan EL. Randomized clinical trials and observational studies: guidelines for assessing respective strengths and limitations. JACC Cardiovasc Interv. 2008;1:211-217.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 168]  [Cited by in F6Publishing: 179]  [Article Influence: 11.9]  [Reference Citation Analysis (0)]
19.  Mann CJ. Observational research methods. Research design II: cohort, cross sectional, and case-control studies. Emerg Med J. 2003;20:54-60.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 723]  [Cited by in F6Publishing: 670]  [Article Influence: 31.9]  [Reference Citation Analysis (0)]