Skip to main content

Advertisement

SpringerLink
Log in

An Analytic Framework for Aligning Observational and Randomized Trial Data: Application to Postmenopausal Hormone Therapy and Coronary Heart Disease

  • Published:
Statistics in Biosciences Aims and scope Submit manuscript

Abstract

We describe a conceptual analytic framework for aligning observational and randomized controlled trial (RCT) data. The framework allows one to (1) use observational data to estimate treatment effects comparable to their RCT counterparts, (2) properly include early events that occur soon after treatment initiation in the analysis of observational data, (3) estimate various treatment effects that are of clinical and scientific relevance while appropriately adjusting for time-varying confounders in both the RCT and observational analyses, (4) assess the generalizability of RCT findings in the more diverse populations generally found in the observational data, and (5) combine both types of data to study associations that cannot be addressed by one study or a single data set. We describe the theoretical application of this framework to the Women’s Health Initiative data to examine the relation between postmenopausal hormone therapy and coronary heart disease. The analytic framework can be tailored to specific exposure-outcome associations and data sources, and may be refined as more is learned about its strengths and limitations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Allison MA, Manson JE (2006) Observational studies and clinical trials of menopausal hormone therapy: can they both be right? Menopause 13(1):1–3

    Article  Google Scholar 

  2. Arbogast PG, Ray WA (2009) Use of disease risk scores in pharmacoepidemiologic studies. Stat Methods Med Res 18(1):67–80

    Article  MathSciNet  Google Scholar 

  3. Barrett-Connor E (2007) Hormones and heart disease in women: the timing hypothesis. Am J Epidemiol 166(5):506–510

    Article  Google Scholar 

  4. Barrett-Connor E, Grady D (1998) Hormone replacement therapy, heart disease, and other considerations. Annu Rev Public Health 19:55–72

    Article  Google Scholar 

  5. Benson K, Hartz AJ (2000) A comparison of observational studies and randomized, controlled trials. N Engl J Med 342(25):1878–1886

    Article  Google Scholar 

  6. Cain LE, Robins JM, Lanoy E, Logan R, Costagliola D, Hernán MA (2010) When to start treatment? A systematic approach to the comparison of dynamic regimes using observational data. Int J Biostat 6(2):18

    MathSciNet  Google Scholar 

  7. Col NF, Pauker SG (2003) The discrepancy between observational studies and randomized trials of menopausal hormone therapy: did expectations shape experience? Ann Intern Med 139(11):923–929

    Article  Google Scholar 

  8. Cole SR, Stuart EA (2010) Generalizing evidence from randomized clinical trials to target populations: the ACTG 320 trial. Am J Epidemiol 172(1):107–115

    Article  Google Scholar 

  9. Concato J, Shah N, Horwitz RI (2000) Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 342(25):1887–1892

    Article  Google Scholar 

  10. Cotter D, Zhang Y, Thamer M, Kaufman J, Hernán MA (2008) The effect of epoetin dose on hematocrit. Kidney Int 73(3):347–353

    Article  Google Scholar 

  11. Creasman WT, Hoel D, Disaia PJ (2003) WHI: Now that the dust has settled: a commentary. Am J Obstet Gynecol 189(3):621–626

    Article  Google Scholar 

  12. Danaei G, Garcia Rodriguez LA, Cantero OF, Logan R, Hernan MA (2011) Observational data for comparative effectiveness research: an emulation of randomised trials of statins and primary prevention of coronary heart disease. Stat Methods Med Res. doi:10.1177/0962280211403603

    Google Scholar 

  13. Diehr P, Yanez D, Ash A, Hornbrook M, Lin DY (1999) Methods for analyzing health care utilization and costs. Annu Rev Public Health 20:125–144

    Article  Google Scholar 

  14. Eichler HG, Abadie E, Breckenridge A, Flamion B, Gustafsson LL, Leufkens H, Rowland M, Schneider CK, Bloechl-Daum B (2011) Bridging the efficacy-effectiveness gap: a regulator’s perspective on addressing variability of drug response. Nat Rev Drug Discov 10(7):495–506

    Article  Google Scholar 

  15. Federal Coordinating Council for Comparative Effectiveness Research (2009) Report to the President and Congress on comparative effectiveness research. Department of Health and Human Services

  16. Garbe E, Suissa S (2004) Hormone replacement therapy and acute coronary outcomes: methodological issues between randomized and observational studies. Hum Reprod 19(1):8–13

    Article  Google Scholar 

  17. Golder S, Loke YK, Bland M (2011) Meta-analyses of adverse effects data derived from randomised controlled trials as compared to observational studies: methodological overview. PLoS Med 8(5):e1001026

    Article  Google Scholar 

  18. Grady D, Rubin SM, Petitti DB, Fox CS, Black D, Ettinger B, Ernster VL, Cummings SR (1992) Hormone therapy to prevent disease and prolong life in postmenopausal women. Ann Intern Med 117(12):1016–1037

    Article  Google Scholar 

  19. Grimes DA, Lobo RA (2002) Perspectives on the Women’s Health Initiative trial of hormone replacement therapy. Obstet Gynecol 100(6):1344–1353

    Article  Google Scholar 

  20. Grodstein F, Clarkson TB, Manson JE (2003) Understanding the divergent data on postmenopausal hormone therapy. N Engl J Med 348(7):645–650

    Article  Google Scholar 

  21. Hernán MA, Alonso A, Logan R, Grodstein F, Michels KB, Willett WC, Manson JE, Robins JM (2008) Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease. Epidemiology 19(6):766–779

    Article  Google Scholar 

  22. Hernán MA, Brumback B, Robins JM (2000) Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. Epidemiology 11(5):561–570

    Article  Google Scholar 

  23. Hernán MA, Lanoy E, Costagliola D, Robins JM (2006) Comparison of dynamic treatment regimes via inverse probability weighting. Basic Clin Pharmacol Toxicol 98(3):237–242

    Article  Google Scholar 

  24. Hernán MA, Robins JM, García Rodríguez LA (2005) Discussion of Statistical issues arising in the Women’s Health Initiative by Prentice RL, Pettinger M, Andreson GL. Biometrics 61(4):922–930

    Article  MathSciNet  Google Scholar 

  25. Hsia J, Langer RD, Manson JE, Kuller L, Johnson KC, Hendrix SL, Pettinger M, Heckbert SR, Greep N, Crawford S, Eaton CB, Kostis JB, Caralis P, Prentice R (2006) Conjugated equine estrogens and coronary heart disease: the Women’s Health Initiative. Arch Intern Med 166(3):357–365

    Article  Google Scholar 

  26. Humphrey LL, Chan BK, Sox HC (2002) Postmenopausal hormone replacement therapy and the primary prevention of cardiovascular disease. Ann Intern Med 137(4):273–284

    Article  Google Scholar 

  27. Institute of Medicine (2009) Initial national priorities for comparative effectiveness research. The National Academies Press, Washington

    Google Scholar 

  28. Kurth T, Walker AM, Glynn RJ, Chan KA, Gaziano JM, Berger K, Robins JM (2006) Results of multivariable logistic regression, propensity matching, propensity adjustment, and propensity-based weighting under conditions of nonuniform effect. Am J Epidemiol 163(3):262–270

    Article  Google Scholar 

  29. Kuss O, Legler T, Borgermann J (2011) Treatments effects from randomized trials and propensity score analyses were similar in similar populations in an example from cardiac surgery. J Clin Epidemiol 64(10):1076–1084

    Article  Google Scholar 

  30. Liang KY, Zeger SL (1986) Longitudinal data analysis using generalized linear models. Biometrika 73(1):13–22

    Article  MathSciNet  MATH  Google Scholar 

  31. Machens K, Schmidt-Gollwitzer K (2003) Issues to debate on the Women’s Health Initiative (WHI) study. Hormone replacement therapy: an epidemiological dilemma? Hum Reprod 18(10):1992–1999

    Article  Google Scholar 

  32. Manson JE, Allison MA, Rossouw JE, Carr JJ, Langer RD, Hsia J, Kuller LH, Cochrane BB, Hunt JR, Ludlam SE, Pettinger MB, Gass M, Margolis KL, Nathan L, Ockene JK, Prentice RL, Robbins J, Stefanick ML (2007) Estrogen therapy and coronary-artery calcification. N Engl J Med 356(25):2591–2602

    Article  Google Scholar 

  33. Manson JE, Bassuk SS (2007) Invited commentary: hormone therapy and risk of coronary heart disease why renew the focus on the early years of menopause? Am J Epidemiol 166(5):511–517

    Article  Google Scholar 

  34. Manson JE, Hsia J, Johnson KC, Rossouw JE, Assaf AR, Lasser NL, Trevisan M, Black HR, Heckbert SR, Detrano R, Strickland OL, Wong ND, Crouse JR, Stein E, Cushman M (2003) Estrogen plus progestin and the risk of coronary heart disease. N Engl J Med 349(6):523–534

    Article  Google Scholar 

  35. McDonough PG (2002) The randomized world is not without its imperfections: reflections on the Women’s Health Initiative Study. Fertil Steril 78(5):951–956

    Article  Google Scholar 

  36. Mendelsohn ME, Karas RH (2007) HRT and the young at heart. N Engl J Med 356(25):2639–2641

    Article  Google Scholar 

  37. Michels KB, Manson JE (2003) Postmenopausal hormone therapy: a reversal of fortune. Circulation 107(14):1830–1833

    Article  Google Scholar 

  38. Mikkola TS, Clarkson TB (2002) Estrogen replacement therapy, atherosclerosis, and vascular function. Cardiovasc Res 53(3):605–619

    Article  Google Scholar 

  39. Murphy SA (2003) Optimal dynamic treatment regimes. J R Stat Soc B 65(2):331–355

    Article  MATH  Google Scholar 

  40. Naftolin F, Taylor HS, Karas R, Brinton E, Newman I, Clarkson TB, Mendelsohn M, Lobo RA, Judelson DR, Nachtigall LE, Heward CB, Hecht H, Jaff MR, Harman SM (2004) The Women’s Health Initiative could not have detected cardioprotective effects of starting hormone therapy during the menopausal transition. Fertil Steril 81(6):1498–1501

    Article  Google Scholar 

  41. Nallamothu BK, Hayward RA, Bates ER (2008) Beyond the randomized clinical trial: the role of effectiveness studies in evaluating cardiovascular therapies. Circulation 118(12):1294–1303

    Article  Google Scholar 

  42. North American Menopause Society (2012) The 2012 hormone therapy position statement of the North American Menopause Society. Menopause 19(3):257–271

    Google Scholar 

  43. Phillips LS, Langer RD (2005) Postmenopausal hormone therapy: critical reappraisal and a unified hypothesis. Fertil Steril 83(3):558–566

    Article  Google Scholar 

  44. Platt R, Carnahan RM, Brown JS, Chrischilles E, Curtis LH, Hennessy S, Nelson JC, Racoosin JA, Robb M, Schneeweiss S, Toh S, Weiner MG (2012) The U.S. food and drug administration’s mini-sentinel program: status and direction. Pharmacoepidemiol Drug Saf 21(Suppl 1):1–8

    Google Scholar 

  45. Prentice RL, Langer R, Stefanick ML, Howard BV, Pettinger M, Anderson G, Barad D, Curb JD, Kotchen J, Kuller L, Limacher M, Wactawski-Wende J (2005) Combined postmenopausal hormone therapy and cardiovascular disease: toward resolving the discrepancy between observational studies and the Women’s Health Initiative clinical trial. Am J Epidemiol 162(5):404–414

    Article  Google Scholar 

  46. Prentice RL, Langer RD, Stefanick ML, Howard BV, Pettinger M, Anderson GL, Barad D, Curb JD, Kotchen J, Kuller L, Limacher M, Wactawski-Wende J (2006) Combined analysis of Women’s Health Initiative observational and clinical trial data on postmenopausal hormone treatment and cardiovascular disease. Am J Epidemiol 163(7):589–599

    Article  Google Scholar 

  47. Prentice RL, Manson JE, Langer RD, Anderson GL, Pettinger M, Jackson RD, Johnson KC, Kuller LH, Lane DS, Wactawski-Wende J, Brzyski R, Allison M, Ockene J, Sarto G, Rossouw JE (2009) Benefits and risks of postmenopausal hormone therapy when it is initiated soon after menopause. Am J Epidemiol 170(1):12–23

    Article  Google Scholar 

  48. Psaty BM, Siscovick DS (2010) Minimizing bias due to confounding by indication in comparative effectiveness research: the importance of restriction. JAMA 304(8):897–898

    Article  Google Scholar 

  49. Ray WA (2003) Evaluating medication effects outside of clinical trials: new-user designs. Am J Epidemiol 158(9):915–920

    Article  Google Scholar 

  50. Robins JM (1986) A new approach to causal inference in mortality studies with sustained exposure periods—application to control of the health worker survivor effect. Math Model 7:1393–1512

    Article  MathSciNet  MATH  Google Scholar 

  51. Robins JM (1989) The analysis of randomized and non-randomized AIDS treatment trials using a new approach to causal inference in longitudinal studies. In: Sechrest L, Freeman H, Mulley A (eds) Health service research methodology: a focus on AIDS. NCHSR, US Public Health Service, Washington, pp 113–159

    Google Scholar 

  52. Robins JM (1993) Analytic methods for estimating HIV treatment and cofactor effects. In: Ostrow DG, Kessler R (eds) Methodological issues of AIDS mental health research. Plenum, New York, pp 213–290

    Google Scholar 

  53. Robins JM, Hernán MA (2008) Estimation of the causal effects of time-varying exposures. In: Fitzmaurice G, Davidian M, Verbeke G, Molenberghs G (eds) Longitudinal data analysis. Chapman and Hall/CRC Press, New York, pp 553–599

    Chapter  Google Scholar 

  54. Robins JM, Hernán MA, Brumback B (2000) Marginal structural models and causal inference in epidemiology. Epidemiology 11(5):550–560

    Article  Google Scholar 

  55. Robins JM, Hernán MA, Siebert U (2004) Effects of multiple interventions. In: Ezzati M, Lopez AD, Murray CJL (eds) Comparative quatification of health risks: global and regional burden of disease attributable to selected major risk factors. World Health Organization, Geneva

    Google Scholar 

  56. Rosenbaum PR, Rubin DB (1983) The central role of the propensity score in observational studies for causal effects. Biometrika 70:41–55

    Article  MathSciNet  MATH  Google Scholar 

  57. Rossouw JE (2005) Coronary heart disease in menopausal women: implications of primary and secondary prevention trials of hormones. Maturitas 51(1):51–63

    Article  Google Scholar 

  58. Rossouw JE, Prentice RL, Manson JE, Wu L, Barad D, Barnabei VM, Ko M, LaCroix AZ, Margolis KL, Stefanick ML (2007) Postmenopausal hormone therapy and risk of cardiovascular disease by age and years since menopause. JAMA 297(13):1465–1477

    Article  Google Scholar 

  59. Rothman KJ, Greenland S, Lash TL (eds) (2008) Modern epidemiology, 3rd edn. Lippincott Williams & Wilkins, Philadelphia

    Google Scholar 

  60. Santen RJ, Allred DC, Ardoin SP, Archer DF, Boyd N, Braunstein GD, Burger HG, Colditz GA, Davis SR, Gambacciani M, Gower BA, Henderson VW, Jarjour WN, Karas RH, Kleerekoper M, Lobo RA, Manson JE, Marsden J, Martin KA, Martin L, Pinkerton JV, Rubinow DR, Teede H, Thiboutot DM, Utian WH (2010) Postmenopausal hormone therapy: an endocrine society scientific statement. J Clin Endocrinol Metab 95:s1–s66

    Article  Google Scholar 

  61. Schneeweiss S, Patrick AR, Sturmer T, Brookhart MA, Avorn J, Maclure M, Rothman KJ, Glynn RJ (2007) Increasing levels of restriction in pharmacoepidemiologic database studies of elderly and comparison with randomized trial results. Med Care 45(10 Supl 2):S131–142

    Article  Google Scholar 

  62. Solomon DH, Avorn J, Sturmer T, Glynn RJ, Mogun H, Schneeweiss S (2006) Cardiovascular outcomes in new users of coxibs and nonsteroidal antiinflammatory drugs: high-risk subgroups and time course of risk. Arthritis Rheumatol 54(5):1378–1389

    Article  Google Scholar 

  63. Sorensen HT, Lash TL, Rothman KJ (2006) Beyond randomized controlled trials: a critical comparison of trials with nonrandomized studies. Hepatology 44(5):1075–1082

    Article  Google Scholar 

  64. Stampfer M (2004) Commentary: Hormones and heart disease: do trials and observational studies address different questions? Int J Epidemiol 33(3):454–455

    Article  Google Scholar 

  65. Stampfer MJ, Colditz GA (1991) Estrogen replacement therapy and coronary heart disease: a quantitative assessment of the epidemiologic evidence. Prev Med 20(1):47–63

    Article  Google Scholar 

  66. Sturdee DW, Pines A, Archer DF, Baber RJ, Barlow D, Birkhauser MH, Brincat M, Cardozo L, de Villiers TJ, Gambacciani M, Gompel AA, Henderson VW, Kluft C, Lobo RA, MacLennan AH, Marsden J, Nappi RE, Panay N, Pickar JH, Robinson D, Simon J, Sitruk-Ware RL, Stevenson JC (2011) Updated IMS recommendations on postmenopausal hormone therapy and preventive strategies for midlife health. Climacteric 14(3):302–320

    Article  Google Scholar 

  67. Tannen RL, Weiner MG, Xie D, Barnhart K (2007) A simulation using data from a primary care practice database closely replicated the women’s health initiative trial. J Clin Epidemiol 60(7):686–695

    Article  Google Scholar 

  68. Taubman SL, Robins JM, Mittleman MA, Hernan MA (2009) Intervening on risk factors for coronary heart disease: an application of the parametric g-formula. Int J Epidemiol 38(6):1599–1611

    Article  Google Scholar 

  69. The Women’s Health Initiative Study Group (1998) Design of the Women’s Health Initiative clinical trial and observational study. Control Clin Trials 19(1):61–109

    Article  Google Scholar 

  70. The Writing Group for the Women’s Health Initiative Investigators (2002) Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women’s Health Initiative randomized controlled trial. JAMA 288(3):321–333

    Article  Google Scholar 

  71. Thompson WA Jr (1977) On the treatment of grouped observations in life studies. Biometrics 33(3):463–470

    Article  MathSciNet  MATH  Google Scholar 

  72. Toh S, Hernán MA (2008) Causal inference from longitudinal studies with baseline randomization. Int J Biostat 4(1):22

    MathSciNet  Google Scholar 

  73. Toh S, Hernández-Díaz S, Logan R, Robins JM, Hernán MA (2010) Estimating absolute risks in the presence of nonadherence: an application to a follow-up study with baseline randomization. Epidemiology 21(4):528–539

    Article  Google Scholar 

  74. Toh S, Hernández-Díaz S, Logan R, Rossouw JE, Hernán MA (2010) Coronary heart disease in postmenopausal recipients of estrogen plus progestin therapy: does the increased risk ever disappear? A randomized trial. Ann Intern Med 152(4):211–217

    Article  Google Scholar 

  75. van der Laan M, Petersen ML, Joffe MM (2005) History-adjusted marginal structural models and statically-optimal dynamic treatment regimens. Int J Biostat 1:4

    MathSciNet  Google Scholar 

  76. Vandenbroucke JP (2004) When are observational studies as credible as randomised trials? Lancet 363(9422):1728–1731

    Article  Google Scholar 

  77. Wasserman L (2006) All of nonparametric statistics. Springer, New York

    MATH  Google Scholar 

  78. Young JG, Cain LE, Robins JM, O’Reilly EJ, Hernán MA (2011) Comparative effectiveness of dynamic treatment regimes: an application of the parametric g-formula. Stat Biosci 2011(3):119–143

    Article  Google Scholar 

Download references

Acknowledgements

Dr. Toh is partially supported by the Agency for Healthcare and Research Quality (R03HS019024). Dr. Manson is partially supported by the National Heart, Lung, and Blood Institute (R01HL034594, R01HL088521, and N0I-WH32109).

Author information

Authors and Affiliations

  1. Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, 133 Brookline Avenue 6th Floor, Boston, MA, 02215, USA

    Sengwee Toh

  2. Division of Preventive Medicine, Brigham and Women’s Hospital, Harvard Medical School, 900 Commonwealth Avenue East 3rd Floor, Boston, MA, 02215, USA

    Joann E. Manson

Authors
  1. Sengwee Toh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joann E. Manson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sengwee Toh.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Toh, S., Manson, J.E. An Analytic Framework for Aligning Observational and Randomized Trial Data: Application to Postmenopausal Hormone Therapy and Coronary Heart Disease. Stat Biosci 5, 344–360 (2013). https://doi.org/10.1007/s12561-012-9073-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12561-012-9073-6

Keywords

Search

Navigation