Skip to main navigation menu Skip to main content Skip to site footer
##common.pageHeaderLogo.altText##

Review article: Biomedical intelligence

Vol. 142 No. 0910 (2012)

Meta-analyses: what they can and cannot do

  • Alain J. Nordmann
  • Benjamin Kasenda
  • Matthias Briel
DOI
https://doi.org/10.4414/smw.2012.13518
Cite this as:
Swiss Med Wkly. 2012;142:w13518
Published
26.02.2012

Summary

Meta-analyses overcome the limitation of small sample sizes or rare outcomes by pooling results from a number of individual studies to generate a single best estimate. As long as a meta-analysis is not limited by poor quality of included trials, unexplainable heterogeneity and/or reporting bias of individual trials, meta-analyses can be instrumental in reliably demonstrating benefit or harm of an intervention when results of individual randomised controlled trials are conflicting or inconclusive. Therefore meta-analyses should be conducted as part of a systematic review, i.e., a systematic approach to answer a focused clinical question. Important features of a systematic review are a comprehensive, reproducible search for primary studies, selection of studies using clear and transparent eligibility criteria, standardised critical appraisal of studies for quality, and investigation of heterogeneity among included studies.

Cumulative meta-analysis may prevent delays in the introduction of effective treatments and may allow for early detection of harmful effects of interventions. As opposed to meta-analysis based on aggregate study data, individual patient data meta-analyses offer the advantage to use standardised criteria across trials and reliably investigate subgroup effects of interventions. Network meta-analysis allows the integration of data from direct and indirect comparisons in order to compare multiple treatments in a comprehensive analysis and determine the best treatment among several options.

We conclude that meta-analysis has become a popular, versatile, and powerful tool. If rigorously conducted as part of a systematic review, it is essential for evidence-based decision making in clinical practice as well as on the health policy level.

References

  1. Plackett RL. Studies in the history of probability and statistics: VII. The principle of the arithmetic mean. Biometrika. 1958;45:130–5.
  2. Pearson K. Report on certain enteric fever inoculation statistics. Br Med J. 1904;3:1243–6.
  3. Glass GV. Primary, secondary and meta-analysis of research. Educ Res. 1976;5:3–8.
  4. Cochrane AL. 1931–1971: a critical review, with particular reference to the medical profession. In: Medicines for the Year 2000; London: Office of Health Economics 1979.
  5. Mulrow CD. The medical review article: state of the science. Ann Intern Med. 1987;106:485–8.
  6. Bero L, Rennie D. The Cochrane Collaboration. Preparing, maintaining, and disseminating systematic reviews of the effects of health care. JAMA. 1995;274:1935–8.
  7. Patsopoulos NA, Analatos AA, Ioannidis JP. Relative citation impact of various study designs in the health sciences. JAMA. 2005;293:2362–6.
  8. Tang BM, Eslick GD, Nowson C, Smith C, Bensoussan A. Use of calcium or calcium in combination with vitamin D supplementation to prevent fractures and bone loss in people aged 50 years and older: a meta-analysis. Lancet. 2007;370:657–66.
  9. Bolland MJ, Barber PA, Doughty RN, et al. Vascular events in healthy older women receiving calcium supplementation: randomised controlled trial. BMJ. 2008;336:262–6.
  10. Prince RL, Devine A, Dhaliwal SS, Dick IM. Effects of calcium supplementation on clinical fracture and bone structure: results of a 5-year, double-blind, placebo-controlled trial in elderly women. Arch Intern Med. 2006;166:869–75.
  11. Baron JA, Beach M, Mandel JS, et al. Calcium supplements for the prevention of colorectal adenomas. Calcium Polyp Prevention Study Group. N Engl J Med. 1999;340:101–7.
  12. McAlister FA, Clark HD, van WC, et al. The medical review article revisited: has the science improved? Ann Intern Med. 1999;131:947–51.
  13. Glasziou P, Irwig L, Brain C, Colditz G. Systematic reviews in health care. A practical guide. Cambridge: Cambridge University Press 2001.
  14. Higgins JPT, Green Se. Cochrane handbook of Systematic Reviews of interventions 2011. http://www.cochrane-handbook.org (Accessed October 10, 2011) 2011.
  15. Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26:91–108.
  16. Bravata DM, Olkin I. Simple pooling versus combining in meta-analysis. Eval Health Prof. 2001;24:218–30.
  17. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327:557–60.
  18. Glasziou PP, Sanders SL. Investigating causes of heterogeneity in systematic reviews. Stat Med. 2002;21:1503–11.
  19. Lau J, Antman EM, Jimenez-Silva J, et al. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med. 1992;327:248–54.
  20. LeLorier J, Gregoire G, Benhaddad A, Lapierre J, Derderian F. Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N Engl J Med. 1997;337:536–42.
  21. Briel M, Lane M, Montori VM, et al. Stopping randomized trials early for benefit: a protocol of the Study Of Trial Policy Of Interim Truncation-2 (STOPIT-2). Trials. 2009;10:49.
  22. Wood L, Egger M, Gluud LL, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008;336:601–5.
  23. Dechartres A, Boutron I, Trinquart L, Charles P, Ravaud P. Single-center trials show larger treatment effects than multicenter trials: evidence from a meta-epidemiologic study. Ann Intern Med. 2011;155:39–51.
  24. Juni P, Nartey L, Reichenbach S, et al. Risk of cardiovascular events and rofecoxib: cumulative meta-analysis. Lancet. 2004;364:2021–9.
  25. McGuinness B, O’Hare J, Craig D, et al. Statins for the treatment of dementia. Cochrane Database Syst Rev 2010;CD007514.
  26. Thompson SG, Higgins JP. How should meta-regression analyses be undertaken and interpreted? Stat Med. 2002;21:1559–73.
  27. Briel M, Meade M, Mercat A, et al. Higher vs lower positive end-expiratory pressure in patients with acute lung injury and acute respiratory distress syndrome: systematic review and meta-analysis. JAMA. 2010;303:865–73.
  28. Caldwell DM, Ades AE, Higgins JP. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ. 2005;331:897–900.
  29. Cipriani A, Furukawa TA, Salanti G, et al. Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis. Lancet. 2009;373:746–58.
  30. Glenny AM, Altman DG, Song F, et al. Indirect comparisons of competing interventions. Health Technol Assess. 2005;9:1-iv.
  31. Song F, Loke YK, Walsh T, et al. Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. BMJ. 2009;338:b1147.
  32. Higgins JP, Altman DG, Gotzsche PC, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.
  33. Ioannidis JP, Haidich AB, Pappa M, et al. Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA. 2001;286:821–30.
  34. Colditz GA, Brewer TF, Berkey CS, et al. Efficacy of BCG vaccine in the prevention of tuberculosis. Meta-analysis of the published literature. JAMA. 1994;271:698–702.
  35. Goodman S, Dickersin K. Metabias: a challenge for comparative effectiveness research. Ann Intern Med. 2011;155:61–2.
  36. Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA. 1990;263:1385–9.
  37. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358:252–60.
  38. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151:264–9, W64.
  39. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100.
  40. Egger M, Davey SG, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315:629–34.
  41. Sterne JA, Sutton AJ, Ioannidis JP, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011;343:d4002.
  42. Yusuf S, Teo K, Woods K. Intravenous magnesium in acute myocardial infarction. An effective, safe, simple, and inexpensive intervention. Circulation. 1993;87:2043–6.
  43. ISIS-4: a randomised factorial trial assessing early oral captopril, oral mononitrate, and intravenous magnesium sulphate in 58,050 patients with suspected acute myocardial infarction. ISIS-4 (Fourth International Study of Infarct Survival) Collaborative Group. Lancet. 1995;345:669–85.
  44. Egger M, Smith GD. Misleading meta-analysis. BMJ. 1995;310:752–4.
  45. Bassler D, Briel M, Montori VM, et al. Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis. JAMA. 2010;303:1180–7.
  46. Bolland MJ, Avenell A, Baron JA, et al. Effect of calcium supplements on risk of myocardial infarction and cardiovascular events: meta-analysis. BMJ. 2010;341:c3691.
  47. Bolland MJ, Grey A, Avenell A, Gamble GD, Reid IR. Calcium supplements with or without vitamin D and risk of cardiovascular events: reanalysis of the Women’s Health Initiative limited access dataset and meta-analysis. BMJ. 2011;342:d2040.
  48. Briel M, Bucher HC, Boscacci R, Furrer H. Adjunctive corticosteroids for Pneumocystis jiroveci pneumonia in patients with HIV-infection. Cochrane Database Syst Rev 2006;3:CD006150.

Most read articles by the same author(s)