Skip to main content
SpringerLink
Log in

Sample Size Determination for Cost-Effectiveness Trials

  • Practical Application
  • Sample Size for Cost-Effectiveness Trials
  • Published:
PharmacoEconomics Aims and scope Submit manuscript

Abstract

Methods for determining sample size requirements for cost-effectiveness studies are reviewed and illustrated. Traditional methods based on tests of hypothesis and power arguments are given for the incremental costeffectiveness ratio and incremental net benefit (INB). In addition, a full Bayesian approach using decision theory to determine optimal sample size is given for INB. The full Bayesian approach, based on the value of information, is proposed in reaction to concerns that traditional methods rely on arbitrarily chosen error probabilities and an ill-defined notion of the smallest clinically important difference. Furthermore, the results of cost-effectiveness studies are used for decision making (e.g. should a new intervention be adopted or the old one retained), and employing decision theory, which permits optimal use of current information and the optimal design of new studies, provides a more consistent approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Table I
Fig. 2
Table II
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. O’Brien BJ, Drummond MF, Labelle RJ, et al. In search of power and significance: issues in the design and analysis of stochastic cost-effectiveness studies in heath care. Med Care 1994; 32: 150–63

    Article  PubMed  Google Scholar 

  2. Mullahy J, Manning W. Statistical issues of cost-effectiveness analysis. In: Sloan F, editor. Valuing health care. Cambridge: Cambridge University Press, 1994

  3. van Hout BA, Al MJ, Gordon GS, et al. Costs, effects and C/E ratios alongside a clinical trial. Health Econ 1994; 3: 309–19

    Article  PubMed  Google Scholar 

  4. Wakker P, Klaassen MP. Confidence intervals for cost/effectiveness ratios. Health Econ 1995; 4 (5): 373–81

    Article  PubMed  CAS  Google Scholar 

  5. Willan AR, O’Brien BJ. Confidence intervals for cost-effectiveness ratios: an application of Fieller’s theorem. Health Econ 1996; 5: 297–305

    Article  PubMed  CAS  Google Scholar 

  6. Chaudhary MA, Stearns SC. Confidence intervals for cost-effectiveness ratios: an example from a randomized trial. Stat Med 1996; 15: 1447–58

    Article  PubMed  CAS  Google Scholar 

  7. Mullahy J. What you don’t know can’t hurt you? Statistical issues and standards for medical technology evaluation. Med Care 1996; 34 (12 Suppl.): DS124–35

    Google Scholar 

  8. Manning WG, Fryback DG, Weinstein MC. Reflecting uncertainty in cost effectiveness analysis. In: Gold MR, Siegel JE, Russell LB, et al., editors. Cost effectiveness in health and medicine. New York: Oxford University Press, 1996

    Google Scholar 

  9. Briggs AH, Wonderling DE, Mooney CZ. Pulling cos-teffectiveness analysis up by its bootstraps: a nonparametric approach to confidence interval estimation.Health Econ 1997; 6: 327–40

    Article  PubMed  CAS  Google Scholar 

  10. Polsky D, Glick HA, Willke R, et al. Confidence intervals for cost-effectiveness ratios: a comparison of four methods. Health Econ 1997; 6: 243–52

    Article  PubMed  CAS  Google Scholar 

  11. Severens JL, De Boo TM, Konst EM. Uncertainty of incremental cost-effectiveness ratios. Int J Technol Assess Health Care 1999; 15: 608–14

    Article  PubMed  CAS  Google Scholar 

  12. Briggs AH, Fenn P. Confidence intervals or surfaces? Uncertainty on the cost-effectiveness plane.Health Econ 1998; 7: 723–40

    CAS  Google Scholar 

  13. Briggs AH, Gray AM. Power and sample size calculations for cost-effectiveness studies. Med Decis Making 1998; 18 (2 Suppl.): S81–92

    Article  Google Scholar 

  14. Laska EM, Meisner M, Siegel C. Power and sample in costeffectiveness analysis. Med Decis Making 1999; 19: 339–43

    Article  PubMed  CAS  Google Scholar 

  15. Briggs AH, Mooney CZ, Wonderling DE. Constructing confidence intervals for cost-effectiveness ratios: an evaluation of parametric and non-parametric techniques using Monte Carlo simulation. Stat Med 1999; 18: 3245–62

    Article  PubMed  CAS  Google Scholar 

  16. Briggs AH, Gray AM. Handling uncertainty in economic evaluations of healthcare interventions. BMJ 1999; 319: 635–8

    Article  PubMed  CAS  Google Scholar 

  17. Heitjan DF, Moskowitz AJ, Whang W. Bayesian estimation of cost-effectiveness ratios from clinical trials. Health Econ 1999; 8: 191–201

    Article  PubMed  CAS  Google Scholar 

  18. Briggs AH. A Bayesian approach to stochastic costeffectiveness analysis. Health Econ 1999; 8: 257–61

    Article  PubMed  CAS  Google Scholar 

  19. Willan AR, O’Brien BJ. Sample size and power issues in estimating incremental cost-effectiveness ratios from clinical trials data. Health Econ 1999; 8: 203–11

    Article  PubMed  CAS  Google Scholar 

  20. Gardiner JC, Huebner M, Jetton J, et al. Power and sample size assessments for test of hypotheses on cost-effectiveness ratios. Health Econ 2000; 9: 227–34

    Article  PubMed  CAS  Google Scholar 

  21. Phelps CE, Mushlin AI. On the (near) equivalence of cost-effectiveness and cost-benefit analysis. Int J Technol Assess Health Care 1991; 7: 12–21

    Article  PubMed  CAS  Google Scholar 

  22. Ament A, Baltussen R. The interpretation of results of economic evaluation: explicating the value of health. Health Econ 1997; 6: 625–35

    Article  PubMed  CAS  Google Scholar 

  23. Stinnett AA, Mallahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making 1998; 18 (2 Suppl.): S68–80

    Article  Google Scholar 

  24. Tambour M, Zethraeus N, Johannesson M. A note on confidence intervals in cost-effectiveness analysis. Int J Technol Assess Health Care 1998; 14: 467–71

    Article  PubMed  CAS  Google Scholar 

  25. Heithan DF. Fieller’s method and net health benefit. Health Econ 2000; 9: 327–35

    Article  Google Scholar 

  26. Willan AR. Analysis, sample size and power for estimating incremental net benefit from clinical trials data. Control Clin Trials 2001; 22 (3): 228–37

    Article  PubMed  CAS  Google Scholar 

  27. Willan AR, Lin DY. Incremental net benefit in randomized clinical trials. Stat Med 2001; 20: 1563–74

    Article  PubMed  CAS  Google Scholar 

  28. Willan AR, Chen EB, Cook RJ, et al. Incremental net benefit in randomized clinical trials with quality-adjusted survival. Stat Med 2003; 22: 353–62

    Article  PubMed  Google Scholar 

  29. Willan AR, Briggs AH. The statistical analysis of cost-effectiveness data. Chichester: Wiley, 2006

    Book  Google Scholar 

  30. Beaton DE, Bombardier C, Katz JN, et al. Looking for important change/differences in studies of responsiveness. J Rheumatol 2001; 28: 400–5

    PubMed  CAS  Google Scholar 

  31. Redelmeier DA, Guyatt GH, Goldstein RS. On the debate over methods for estimating the clinically important difference. J Clin Epidemiol 1996; 49: 1223–4

    Article  PubMed  CAS  Google Scholar 

  32. Copay AG, Subach BR, Glassman SD, et al. Understanding the minimum clinically important difference: a review of concepts and methods. Spine 2007; 7: 541–6

    Article  Google Scholar 

  33. Lemeshow S, Hosmer Jr DW, Klar J, et al. The adequacy of sample size in health studies. New York: John Wiley and Sons, 1990

    Google Scholar 

  34. Lachin JM. Introduction to sample size determination and power analysis for clinical trials. Control Clin Trials 1981; 2: 93–113

    Article  PubMed  CAS  Google Scholar 

  35. O’Hagan A, Stevens JW. Bayesian assessment of sample size for clinical trials of cost-effectiveness. Med Decis Making 2001; 21 (3): 219–30

    PubMed  Google Scholar 

  36. Willan AR, Cruess AF, Ballantyne M. Argon green vs krypton red laser photocoagulation of extrafoveal choroidal neovascular lesions: three-year results in age-related macular generation. Can J Ophthalmol 1996; 31: 11–7

    PubMed  CAS  Google Scholar 

  37. Hannah ME, Hannah WJ, Hewson SH, et al. Term breech trial: a multicentre international randomised controlled trial of planned caesarean section and planned vaginal birth for breech presentation at term. Lancet 2000; 356: 1375–83

    Article  PubMed  CAS  Google Scholar 

  38. Willan AR. Power function arguments in support of an alternative approach for analyzing management trials. Control Clin Trials 1994; 15: 211–9

    Article  PubMed  CAS  Google Scholar 

  39. Berry DA, Ho C-H. One-sided sequential stopping boundaries for clinical trials: a decision-theoretic approach. Biometrics 1998; 44: 219–27

    Article  Google Scholar 

  40. Gittins J, Pezeshk H. How large should a trial be? Statistician 2000; 49: 177–97

    Google Scholar 

  41. Gittins J, Pezeshk H. A behavioral Bayes method for determining the size of a clinical trial. Drug Inf J 2002; 34: 355–63

    Article  Google Scholar 

  42. Gittins J, Pezeshk H. A decision theoretic approach to sample size determination in clinical trials. J Biopharm Stat 2002; 12: 535–51

    Article  PubMed  CAS  Google Scholar 

  43. Cheng Y, Su F, Berry DA. Choosing sample size for a clinical trial using decision analysis. Biometrika 2003; 90: 923–36

    Article  Google Scholar 

  44. Pezeshk H. Bayesian techniques for sample size determination in clinical trials: a short review. Stat Methods Med Res 2003; 12: 489–504

    Article  PubMed  Google Scholar 

  45. Pezeshk H, Gittins J. Bayesian approaches to determine the number of subsequent users of a new treatment. Stat Methods Med Res 2006; 15: 585–92

    Article  PubMed  Google Scholar 

  46. Hornberger JC, Brown Jr BW, Halpern J. Designing a cost-effective clinical trial. Stat Med 1995; 14: 2249–59

    Article  PubMed  CAS  Google Scholar 

  47. Claxton K, Posnett J. An economic approach to clinical trial design and research priority setting. Health Econ 1996; 5: 513–24

    Article  PubMed  CAS  Google Scholar 

  48. Hornberger J, Eghtesady P. The cost-benefit of a randomized trial to a health care organization. Control Clin Trials 1998; 19: 198–211

    Article  PubMed  CAS  Google Scholar 

  49. Claxton K. The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. J Health Econ 1999; 18: 341–64

    Article  PubMed  CAS  Google Scholar 

  50. Halpern J, Brown Jr BW, Hornberger J. The sample size for a clinical trial: a Bayesian-decision theoretic approach. Stat Med 2001; 20: 841–58

    Article  PubMed  CAS  Google Scholar 

  51. Claxton K, Thompson KM. A dynamic programming approach to the efficient design of clinical trials. J Health Econ 2001; 20: 797–822

    Article  PubMed  CAS  Google Scholar 

  52. Willan AR, Pinto EM. The expected value of information and optimal clinical trial design [published erratum appears in Stat Med 2006; 25 (4): 720]. Stat Med 2005; 24: 1791–806

    Article  PubMed  Google Scholar 

  53. Willan AR. Clinical decision making and the expected value of information. Clin Trials 2007; 4: 279–85

    Article  PubMed  Google Scholar 

  54. Eckermann S, Willan AR. Expected value of information and decision making in HTA. Health Econ 2007; 16: 195–209

    Article  PubMed  Google Scholar 

  55. Eckermann S, Willan AR. Time and EVSI wait for no patient. Value Health 2008; 11: 522–6

    Article  PubMed  Google Scholar 

  56. Eckermann S, Willan AR. The option value of delay in health technology assessment. Med Decis Making 2008; 28: 300–5

    Article  PubMed  Google Scholar 

  57. Kikuchi T, Pezeshk H, Gittins J. A Bayesian cost-benefit approach to the determination of sample size in clinical trials. Stat Med 2008; 27: 68–82

    Article  PubMed  Google Scholar 

  58. Willan AR. Optimal sample size determinations from an industry perspective based on the expected value of information. Clin Trials 2008; 5 (6): 587–94

    Article  PubMed  Google Scholar 

  59. Willan AR, Kowgier ME. Determining optimal sample sizes for multi-stage randomized clinical trials using value of information methods. Clin Trials 2008; 5: 289–300

    Article  PubMed  Google Scholar 

  60. Eckermann S, Willan AR. Globally optimal trial design for local decision making. Health Econ 2009; 18: 203–16

    Article  PubMed  Google Scholar 

  61. Willan AR, Eckermann S. Optimal clinical trial design using value of information methods with imperfect implementation. Health Econ 2010; 19: 549–61

    PubMed  Google Scholar 

  62. Willan AR, Eckermann S. Accounting for between-study variation in value of information methodology. Health Econ. Epub 2011 Sep 1. DOI 10.1002/hec.1781

    Google Scholar 

  63. Grundy PM, Healy MJR, Rees DH. Economic choice of the amount of experimentation. J R Stat Soc Series B Stat Methodol 1956; 18: 32–48

    Google Scholar 

  64. Raiffa H, Schlaifer R. Applied statistical decision theory. Cambridge (MA): Harvard University, Graduate School of Business Administration, 1961

    Google Scholar 

  65. Nixon RM, Thompson SG. Parametric modelling of cost data in medical studies. Stat Med 2004; 23 (8): 1311–31

    Article  PubMed  CAS  Google Scholar 

  66. Briggs A, Nixon R, Dixon S, et al. Parametric modelling of cost data: some simulation evidence. Health Econ 2005; 14 (4): 421–8

    Article  PubMed  Google Scholar 

  67. Chiba N, van Zanten SJ, Sinclair P, et al. Treating Helicobacter pylori infection in primary care patients with uninvestigated dyspepsia: the Canadian Adult Dyspepsia Empiric Treatment-Helicobacter pylori positive (CADET-Hp) randomised controlled trial. BMJ 2002; 324 (7344): 1012–6

    Article  PubMed  CAS  Google Scholar 

  68. Willan AR. Incremental net benefit in the analysis of economic data from clinical trials with application to the CADET-Hp Trial. Eur J Gastroenterol Hepatol 2004; 16 (6): 543–9

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

A.R. Willan is funded by the Discovery Grant Program of the Natural Sciences and Engineering Research Council of Canada (grant number 44868-08). The author is heavily indebted to the reviewers whose insightful comments improved the manuscript immeasurably.

Author information

Authors and Affiliations

  1. SickKids Research Institute and University of Toronto CHES, 555 University Avenue, Toronto, ON, M5G 1X8, Canada

    Andrew R. Willan

Authors
  1. Andrew R. Willan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrew R. Willan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Willan, A.R. Sample Size Determination for Cost-Effectiveness Trials. Pharmacoeconomics 29, 933–949 (2011). https://doi.org/10.2165/11587130-000000000-00000

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.2165/11587130-000000000-00000

Keywords

Search

Navigation