Cost-benefit analysis of the FDA: The case of the prescription drug user fee acts

https://doi.org/10.1016/j.jpubeco.2007.09.010Get rights and content

Abstract

The U.S. Food and Drug Administration (FDA) is estimated to regulate markets accounting for about 20% of consumer spending in the U.S. Despite the FDA's strict adherence to evidence-based evaluation of the safety and efficacy of the products it regulates, there exists no generally agreed upon evidence-based methodology to evaluate the agency's own safety and efficacy record. This paper proposes a methodology to evaluate FDA policies in general, and the central speed-safety tradeoff it faces, in particular. We apply this methodology to estimate the welfare effects of a major piece of legislation affecting this tradeoff, the Prescription Drug User Fee Acts (PDUFA). These acts mandated FDA performance goals in reviewing and acting on drug applications within specified time periods, in return for levying fees on drug manufacturers' submissions. Our methodology uses data on the U.S. sales of drugs as well as the FDA review and withdrawal times for those drugs to estimate measures of the private and social surplus associated with the agency in general, and changes in the speed-safety tradeoff induced by PDUFA, in particular. We find that PDUFA raised the private surplus of producers, and thus innovative returns, by about $7 to $11 billion. Depending on assumptions about the market power of producers during patent protection, we find that PDUFA raised consumer welfare between $7 and $20 billion; thus the combined social surplus was raised by $14 to $31 billion. Converting these economic gains into equivalent health benefits, we find that the more rapid access of drugs on the market enabled by PDUFA saved the equivalent of 140,000 to 310,000 life years. Additionally, we estimate an upper bound on the adverse effects of PDUFA based on drugs submitted during PDUFA I/II and subsequently withdrawn for safety reasons, and find that an extreme upper bound of about 56,000 life years were lost. This estimate is an extreme upper bound as it assumes all withdrawals since the inception of PDUFA were due to PDUFA and that there were no patients who benefitted from the withdrawn drugs. We discuss how our general methodology could be used to perform a quantitative and evidence-based evaluation of the desirability of other FDA policies in the future, particularly those affecting the speed-safety tradeoff of the agency.

Introduction

In virtually all developed countries, regulatory authorities provide public oversight of the safety and efficacy of prescription drugs prior to their being approved for marketing. In the U.S., such oversight is conducted by the Food and Drug Administration (FDA). A central tradeoff facing the FDA, and argued by many to be the most central one, involves balancing two goals — fulfilling its mission set by Congress to assure the safety and efficacy of drugs, while at the same time advancing public health by not slowing down or disabling the innovative process by which new medical products reach the market.

Critics of the FDA, domestic and foreign, appear on both sides of this tradeoff. Some observers have argued that the FDA is not taking enough time to evaluate new drugs, thereby allowing unsafe drugs to be marketed, while others have argued that the agency is taking too long to do so and therefore is inflicting harmful effects on innovative returns and patient welfare.3 However, surprisingly very little quantitative empirical evidence has been put forward to evaluate the degree to which the speed and safety tradeoff facing the FDA is being resolved efficiently. More generally, there seems to be no suggested quantitative methodology or framework for assessing the economic efficiency of the central speed-safety tradeoff of the agency.4 This is somewhat paradoxical, since despite the agency's strict adherence to evidence-based evaluation of the products it oversees, there is less evidence on its own safety and efficacy track record. Put differently, no product application would pass the FDA approval process with the quality and type of evidence that currently exists for evaluating the FDA policies themselves. The welfare consequences of this lack of methodology and systematic evidence may be quite substantial, as the FDA is estimated to regulate markets accounting for about 20% of consumer spending in the U.S.

Motivated by this lack of quantitative assessment of FDA policies, this paper proposes a general methodology to evaluate the common speed-safety tradeoff of FDA regulations. The methodology relies on the most common form of data available surrounding the drug approval process, namely, the distribution of approval and withdrawal times of drugs as well as the distribution of sales of the approved drugs. We use these commonly available data to estimate the social value of approved drugs. As indicated in Fig. 1, in this paper we interpret the overall social value of a drug as the discounted sum of its yearly social welfare from the time of review and approval to the time of withdrawal, if the drug is withdrawn.

As shown in Fig. 1, the annual social surplus occurs after the drug is reviewed and approved, then is split up into consumer and producer surplus components while on the market, and vanishes completely once the drug is withdrawn (if ever). Therefore, if the drug is beneficial, as it is in the figure, its overall social value falls with the review time and rises with the time until withdrawal. However, if the drug is harmful, as when the social surplus is negative and below the x-axis in the figure, then its overall social value rises with the review time and falls with the time until withdrawal. The agency in general, and separate regulations in particular, influence aggregate social welfare by affecting the distribution of review and withdrawal times, as well as the magnitude and signs of the post-approval annual flows of social surplus.

We apply this general framework to quantify the change in aggregate social welfare induced by major legislative acts comprised of the Prescription Drug User Fee Act (PDUFA) of 1992, later continued as PDUFA II in 1997 and PUDFA-III in 2002. The just recent renewal of PDUFA in 2007 occurs after our data time period ends. These legislative acts specified performance goals for the FDA in terms of faster review times, while levying taxes in the form of user fees on the sponsoring applicant for consideration of new and supplemental drug applications, as well as for existing manufacturing establishments and products. The user fee revenues obtained under PDUFA represent a substantial portion of the FDA's total revenues obligated to processing human drug applications, often estimated to be close to half of the funding of the drug review process (U.S. Food and Drug Administration, 2003). The implementation of PDUFA and its impact on the central speed-safety tradeoff of the agency can be envisaged in Fig. 1; faster review of drugs involves a shift to the left of the surplus curve in Fig. 1, which is accomplished with the additional resources generated from the user fees, but with the potential consequence of less attention being devoted to the safety of the drugs approved, entailing a downward shift in the surplus curve of Fig. 1 together with a reduction in withdrawal times.

To estimate the impact of PDUFA on aggregate social welfare, we first assess the impact of these Acts on review times using 662 New Molecular Entity (NME) drug approvals in the years 1979–2002 prior to and following enactment of the Acts in 1992. We find that even though there was a decline in review times of 2% a year prior to PDUFA, passage and implementation of PDUFA I and II accelerated the decline by 6–7% and 3–4% a year respectively. Using the estimated effects of PDUFA on approval times for each of the drug approvals, we are then able to estimate the counterfactual approval time that would have occurred in the absence of PDUFA.

The estimated changes in review times induced by PDUFA are first used to assess the impact on producer surplus or variable profits. One of the main issues of the speed-safety tradeoff facing the FDA is how PDUFA affects innovative returns. The revenues of a drug under PDUFA are derived from actual sales data, and the counterfactual revenues come from delaying the entry of the drug by the predicted drug-specific delay between the observed and counterfactual review times in a world with and without PDUFA. To estimate the effect of PDUFA on the innovative return of the drug, we then net out from the earlier arrival of its sales the application, product, and establishment fees associated with the drug under PDUFA, as well as variable costs estimated by price levels before patents have run out and extensive entry of generic drugs has occurred. Our main finding is that producer surplus rose with the introduction of PDUFA by about $7–$11 billion or 1.2% (at a real discount rate of 3% and measured in 1992 dollars at the inception of PDUFA). For the sample of 284 drugs for which we had sales information during PDUFA I and PDUFA II, this represents a gain of about $25–$39 million per drug launched.5

We then consider estimation of the social surplus from the additional speed induced by PDUFA through adding estimates of consumer surplus to the estimated producer surplus levels. We focus our discussion on two ways of adding consumer surplus to compute the social benefits of PDUFA. The first case occurs when full price discrimination is infeasible so that some consumer surplus exists under the patent. We use as a benchmark the case when consumer surplus is half of producer surplus, as turns out to be true when demand is linear and there are constant returns to scale. Also, under no price discrimination, we derive the simple but plausible conditions under which measures of sales, the most commonly available data on drug usage, constitute a lower bound to the social surplus. The second case occurs when price discrimination is complete so that there is no consumer surplus during the patent period but only after generics have entered. In this case, sales during the patent period represent producer surplus, and sales just prior to expiration represent the consumer surplus after the patent has expired. Our major findings here are that under no price discrimination, the social surplus generated by the greater speed of PDUFA is $24–$31 billion or 1.2% under demand linearity and constant returns to scale, but bounded below by $21 billion or 1.2% under less stringent demand and cost assumptions. For the case of complete price discrimination, we find that the benefit of the additional speed is $14–18 billion or 1.2%. For the 284 drugs with sales data, these estimates amount to a gain of about $85–$109 million (in the case of linear demand and constant returns) or $49–62 million per drug (in the case of complete price discrimination) introduced since the inception of PDUFA.

These evaluations of the social benefits of speed are then compared to the social costs of a possibly less safe approval process. To assess the impact on consumer safety, we first consider the effect PDUFA had on the fraction of drugs withdrawn and how rapidly they were withdrawn. However, these measures of the quantity and timing of withdrawal do not fully capture the quality of the drugs withdrawn. We also compute how much harm to health PDUFA must have imposed in order to offset the gains from speed due to more rapid review. Our major findings are that the proportion and timing of withdrawal of drugs approved pre- and post-PDUFA do not differ in a statistically significant way; about 2–3% of approved drugs are withdrawn at the same speed before and after the Acts. In addition, we compute an extreme upper bound on the adverse safety effects induced by PDUFA by assuming that all NME withdrawals after 1992 were due to PDUFA and that the were no benefits associated with the drugs so that their social surplus is measured by the harmful health effects the withdrawn drugs imposed. Using this extreme upper bound on the adverse safety effects of PDUFA, we find that the drugs approved and withdrawn during PDUFA cost about 56,000 life years as compared to the gains in health implicit in the greater speed generated by PDUFA, which are estimated at the equivalent of 140,000 to 310,000 life years. Our estimates therefore suggest that PDUFA raised ex-post welfare, even without considering the positive effects it may have had on innovation through raising innovative returns.

The paper may be briefly outlined as follows. In Section 2 we provide a general framework for assessing quantitatively the value of changes to the speed-safety tradeoff facing the FDA. In Section 3 we provide a brief overview of PDUFA I and PDUFA II and reports on two previous studies we conducted (Berndt et al., 2005a, Berndt et al., 2005b)6 that we use as one of many inputs into the new cost-benefit analysis discussed here; this previous research did not undertake the more ambitious goal of providing an economic framework for evaluation of PDUFA, the goal of this research. In 4 The effect of PDUFA on producer surplus, 5 The effect of PDUFA on social surplus we discuss estimation of the beneficial speed effects of PDUFA on producer, consumer, and social welfare, using methods of bounding social surplus from sales data. In Section 6 we estimate negative safety effects of PDUFA by considering their effects on drug withdrawals as well as calculating the equivalent losses in social surplus or health that must be induced to offset the observed benefits of speed. Finally, in Section 7 we summarize our findings, note limitations of our research, and suggest directions for further research.

Section snippets

A framework for evaluating changes in the speed-safety tradeoff at the FDA

This section presents a parsimonious framework for evaluating changes in FDA policies that affect the speed-safety tradeoff. Let p(y) denote the inverse demand curve for a drug in a given year and let c(y) be the cost function where y represents output. The annual producer π(y) and consumer surplus s(y) are specified in a standard manner asπ(y)=p(y)yc(y)s(y)=0y[p(q)p(y)]q.

This specification could be modified to other forms of non-canonical producer and consumer surplus relationships,

Background on PDUFA

The concept of payment of user fees by individuals or firms being in exchange for services by a government regulatory body has ample precedent, e.g., application submission fees to the U.S. Patent and Trademark Office. The development of the Prescription Drug User Fee Act permitted the FDA to collect fees from sponsors submitting a New Drug Application (NDA or Biologics License Application (BLA) for review. The passage of PDUFA I in 1992 was, however, somewhat controversial in that the amount

The effect of PDUFA on producer surplus

We now report results from estimating the effects of PDUFA on the total producer surplus ∫ΠdF, which represents the innovative return of the drug after R&D has been undertaken. PDUFA affects an innovator's returns by raising both the costs and the benefits of innovation. The cost is raised by the amount of the user fee taxes levied, while the benefit is raised by the gains in the present value of the innovator's return induced by the more rapid FDA approval.The effect of faster approval on

The effect of PDUFA on social surplus

Next we report results from estimating the effects of PDUFA on the total social surplus ∫WdF by augmenting the estimated effects of PDUFA on producer surplus ∫ΠdF with the estimated effects on consumer surplus ∫SdF. However, in contrast to the producer surplus, the annual consumer surplus is lower during the patent period, when it is limited by the degree of market power exercised by the patent holder, than after the patent has expired, when prices come down and consumer surplus rises. Annual

Estimating the safety effects of PDUFA

The previous discussion reports findings from estimating the first term – the speed component – of the change in aggregate welfare A towards Ao induced by changes due to PDUFA. In this section we discuss the second term representing any offsetting changes in safety that may have occurred. We also attempt to price out these safety deteriorations in terms of health effects.

Discussion, limitations, and concluding remarks

The analysis presented here is a first attempt towards a systematic and quantitative evaluation of the speed-safety tradeoff facing the FDA. As a consequence, it has several shortcomings affecting the precision of the estimates, which also raise important avenues for future research.

First, our analysis ignores the important role of uncertainty in the decision making process, and how the government may or may not resolve that uncertainty better than the private sector. In the beginning, there is

References (32)

  • J.A. DiMasi et al.

    The price of innovation: new estimates of drug development costs

    Journal of Health Economics

    (2003)
  • M.K. Olson

    Are novel drugs more risky for patients than less novel drugs?

    Journal of Health Economics

    (2004)
  • A. Begosh et al.

    Black Box Warnings and Drug Safety: Examining the Determinants and Timing of FDA Warning Labels

  • E.R. Berndt et al.

    Pharmaceutical innovations and market dynamics: tracking effects on price indexes for antidepressant drugs, Brookings papers

    Microeconomics

    (1996)
  • E.R. Berndt et al.

    Industry funding of the FDA: effects of PDUFA on approval times and withdrawal rates

    Nature Reviews: Drug Discovery

    (2005)
  • E.R. Berndt et al.

    Assessing the Impacts of the Prescription Drug User Fee Acts (PDUFA) on the FDA Approval Process, National Bureau of Economic Research

  • E.R. Berndt et al.

    The impact of incremental innovation on biopharmaceuticals: drug utilization in original and supplemental indications

    PharmacoEconomics

    (2006)
  • D.P. Carpenter

    Groups, the media, agency waiting costs, and FDA drug approval

    American Journal of Political Science

    (2002)
  • D.P. Carpenter et al.

    Fundamental Problems in Linear Statistical Analyses of Regulatory Approval Times upon Policy Indicators

  • M. Dixon et al.

    Key factors in the rising cost of new drug discovery and development

    Nature Reviews: Drug Discovery

    (2004)
  • A.M. Garber et al.

    Insurance and Incentives for Medical Innovation, National Bureau of Economic Research

  • H.G. Grabowski et al.

    Brand loyalty, entry, and price competition in pharmaceuticals after the 1984 Drug Act

    Journal of Law and Economics

    (1992)
  • H.G. Grabowski et al.

    Effective patent life in pharmaceuticals

    International Journal of Technology Management

    (2000)
  • Grabowski, H.G., Wang, Y.R., 2006. Do Faster FDA Drug Reviews Adversely Affect Patient Safety? An Analysis of the 1992...
  • H.G. Grabowski et al.

    Returns on research and development for 1990s new drug introductions

    PharmacoEconomics

    (2002)
  • Cited by (31)

    • Regulation of Safety, Efficacy, and Quality

      2014, Encyclopedia of Health Economics
    • Economic analysis of risk and uncertainty induced by health shocks: A review and extension

      2014, Handbook of the Economics of Risk and Uncertainty
      Citation Excerpt :

      An important aspect of regulating the health risks of medical products is that in many countries including the United States, medical products are jointly regulated by agencies such as the FDA, which screens products to ensure they are safe and effective before they are sold, and the tort liability system, which allows patients to sue manufacturers after they have consumed these products. This “dual” aspect of product safety regulations for health risks has been analyzed by Philipson et al. (2009) who argue that one form of regulation may increase costs in the presence of the other. Work by Philipson et al. (2008) considers the dynamic welfare effects of FDA regulation by considering the present value of all future costs and benefits of the products being regulated.

    • The Economics of Health and Health Care, Ninth Edition

      2024, The Economics of Health and Health Care, Ninth Edition
    View all citing articles on Scopus

    For part of the time this research was undertaken, Philipson was on leave from the University of Chicago to the Food and Drug Administration (FDA) and the Centers for Medicaid and Medicare Services (CMS) as Senior Economic Advisor to the Commissioner and Administrator, respectively. Research support to Philipson from The Milken Institute and the George J. Stigler Center for Study of Economy and State at the University of Chicago, to Berndt and Gottschalk from general funds at the Harvard-MIT Division of Health Sciences and Technology, and to Sun from the Bing Center for Health Economics at RAND and the Medical Scientist Training Program at the University of Chicago is gratefully acknowledged. We thank Ed Hass of the FDA and Roy Gross of MIT for their very significant assistance throughout this project. The views and opinions expressed in this paper are those of the authors, and do not necessarily reflect any official positions or policies of the institutions or companies with which they are or have been affiliated.

    1

    Tel.: +1 617 253 2665; fax: +1 617 258 6855.

    2

    Tel.: +1 310 393 0411x6215.

    View full text