ReviewRecent developments in improving signal detection and reducing placebo response in psychiatric clinical trials
Introduction
Signal detection is the ability to differentiate between an effective drug and placebo; that is, to find a treatment effect when one exists (Mallinckrodt et al., 2007). Not surprisingly, poor signal detection has been linked with difficulties in discovering new therapies (Gelenberg et al., 2008). Khan et al. (2003a) reported that in studies of known effective antidepressants 21.1% of the drug-placebo contrasts were statistically significant in trials with high placebo response, compared with 74.2% significant contrasts in trials with low placebo response. Similarly, in recent schizophrenia trials placebo response has increased and signal detection has become more difficult (Kemp et al., 2010).
Placebo response is the change that occurs after administration of placebo and is caused by a study effect plus a placebo effect (Yang et al., 2005). The study effect is the tendency for a patient’s state to be modified solely from participation in a clinical trial and not to a treatment administered therein. The study effect includes such factors as spontaneous improvement, regression to the mean, superior care provided in the trial as compared to that received prior to trial participation, etc. The study effect influences all patients and could be assessed as the change in patients participating in the trial who were not administered a medication.
The placebo effect is the nonspecific, psychological, or psychophysiologic therapeutic effect often attributed, at least in part, to the expectation that improvement will follow the administration of treatment. Therefore the placebo effect is linked to the awareness and acceptance that a potentially effective treatment is received. The placebo effect influences all patients that receive study medication in a trial and could be assessed as the difference in response between placebo-treated patients and patients not administered drug in the same protocol. Placebo response is not limited to efficacy outcomes, although that is the focus here.
Placebo response and signal detection have been active areas of research. Given placebo response is the result of a study effect plus a placebo effect, trial features might be modified to reduce study and placebo effects, thereby reducing placebo response and improving signal detection. Therefore, we summarized recent empirical and theoretical literature on associations of trial design features with placebo response and trial outcome, along with data and analytic considerations.
Section snippets
Percent randomized to placebo
Earlier research noted that studies with fewer treatment arms and studies with flexible vs. fixed dosing more frequently yielded statistically significant differences from placebo (Khan et al., 2003a, Khan et al., 2003b). However, flexible dose studies typically had fewer treatment arms. Hence, the two effects were confounded and the independent contribution of each was unclear.
Recent literature has focused on the role of patient and rater expectation in placebo response (Rutherford et al., 2009
Outcome measures
The psychometric properties of the scales used to assess severity of symptoms have been debated, especially in MDD. For example, Gibbons et al. (1993) noted that the HAMD loses accuracy in assessing changes in depression severity because it does not define an uni-dimensional depressive state. As a result, uni-dimensional factors of the HAMD that focus on the core symptoms of depression have been defined (Bech et al., 2006, Maier and Philipp, 1985, Gibbons et al., 1993, McIntyre et al., 2005;
Discussion
Improving signal detection and reducing placebo response have remained active areas of investigation. This review summarized recent (2007–2010) research on these topics. Focus was on design considerations plausibly linked with patient and rater expectation, or thought to increase placebo response through an increased study effect. Data and analysis considerations potentially influencing precision or bias in estimates of treatment effects were also examined. Many of the studies in this review
Funding
Funding for this study was provided by Eli Lilly and Co. Lilly had no further role in study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the paper for publication.
Contribution
Mallinckrodt was the primary author of the paper. However, Tamura and Tanaka each authored specific sections of the paper. All authors contributed to the review of the literature and interpretation of it, with the magnitude of that work corresponding to the authorship position. All authors have approved the final manuscript.
Conflict of interest
All authors are employees of and share holders in Eli Lilly and Co.
Acknowledgments
No acknowledgments to declare.
References (44)
- et al.
A critical examination of the sensitivity of unidimensional subscales derived from the Hamilton depression rating scale to antidepressant drug effects
Journal of Psychiatric Research
(2002) - et al.
The responsiveness of the Hamilton depression rating scale
J Psychiatr Res.
(2000) - et al.
Exactly what does the Hamilton depression rating scale measure?
Journal of Psychiatric Research
(1993) - et al.
Does the probability of receiving placebo influence clinical trial outcome? A meta-regression of double-blind, randomized clinical trials in MDD
European Neuropsychopharmacology
(2009) - et al.
Meta-analysis of the placebo response in antidepressant trials
Journal of Affective Disorders
(2009) - et al.
Dose-response relationship of duloxetine in placebo-controlled clinical trials in patients with major depressive disorder
Psychopharmacology (Berl)
(2006) “Placebo response in major depression and schizophrenia trials: statistical consideration and design strategies”
(2010)- et al.
A randomized, double-blind, placebo-controlled and active comparator trial of pexacerfont, a corticotropin releasing factor receptor-1 antagonist, in the treatment of generalized anxiety disorder
(December 2008) - et al.
The e_ective sample size and a novel small sample degrees of freedom method
The American Statistician
(2009) - et al.
The double-blind variable placebo lead-in period: results from two antidepressant clinical trials
J Clin Psychopharmacol
(2001)
The problem of the placebo response in clinical trials for psychiatric disorders: culprits, possible remedies, and a novel study design approach
Psychother Psychosom
System and method for reducing the placebo effect in controlled clinical trials
“Enrichment design”
The history and current state of antidepressant clinical trial design: a call to action for proof–of–concept studies
Journal of Clinical Psychiatry
A rating scale for depression
Journal of Neurology, Neurosurgery, and Psychiatry
“Comparison of test statistics for the sequential parallel design”
Statistics in Biopharmaceutical Research
What is causing the reduced drug-placebo difference in recent schizophrenia clinical trials and what can be done about it?
Schizophrenia Bulletin
Missing data in clinical studies
Placebo response and antidepressant clinical trial outcome
Journal of Nervous and Mental Disease
Frequency of positive studies among fixed and flexible dose antidepressant clinical trials: an analysis of the food and drug administration summary basis of approval reports
Neuropsychopharmacology
Sources of unreliability in depression ratings
Journal of Clinical Psychopharmacology
Site versus centralized raters in a clinical depression trial: impact on patient selection and placebo response
Journal of Clinical Psychopharmacology
Cited by (32)
Statistical methods in handling placebo effect
2020, International Review of NeurobiologyCitation Excerpt :However, any deviation from balanced allocation ratios has been shown to result in an increase in placebo response rate for a trial and introduce allocation bias (Enck et al., 2011; Enck, Junne, Klosterhalfen, Zipfel, & Martens, 2010). A balanced randomization allocation ratio permits the most optimal assessment of treatment effects and ensures maximum power for the study (Mallinckrodt, Tamura, & Tanaka, 2011). Maintenance of blind: Effective maintenance of the study blind at the participant and the site level is another factor that substantially improves the ability to detect the efficacy signal in a trial.
Determinants of antidepressant response: Implications for practice and future clinical trials
2018, Journal of Affective DisordersCitation Excerpt :This is in contrast to the findings of Khin et al. (2011) who examined 81 studies conducted in a similar time period and found that the placebo response showed a modest increase over the observation period but the treatment effect clearly diminished, resulting in decreasing drug-placebo separation over time. Our finding that treatment response increased with the proportion of subjects on placebo is consistent with a meta-analysis of depression studies (Papakostas and Fava, 2009), an analysis of a patient registry of antipsychotic trials (Mallinckrodt et al., 2011), a review of trials across psychiatry (Weimer et al., 2015) and has been reported in other areas of medicine as well (Enck et al., 2011). This finding has been attributed to expectancy; if the proportion of subjects on placebo is low then the expectation of both subject and investigator is that a given subject is on active treatment (Enck et al., 2011).
A meta-analysis of randomized, placebo-controlled trials of vortioxetine for the treatment of major depressive disorder in adults
2016, European NeuropsychopharmacologyPlacebo response in antipsychotic trials of patients with acute mania. Results of an individual patient data meta-analysis
2015, European NeuropsychopharmacologyCitation Excerpt :The presence of psychotic features at baseline was defined as a score of 3 (‘flight of ideas; tangentially; difficult to follow; rhyming; echolalia’) or 4 (incoherent; communication impossible) on question 7 of the YMRS or a score of 6 (‘Grandiose or paranoid ideas; Ideas of reference’) or 8 (‘Delusions; Hallucinations’) on question 8 (‘Content’) of the YMRS questionnaire (21). Study characteristics included study year (Agid et al., 2013; Cohen et al., 2010; Gispen-de Wied et al., 2012; Sysko and Walsh, 2007; Yildiz et al., 2011), number of visits per protocol (Cohen et al., 2010; Montgomery, 1999), number of study arms (Agid et al., 2013), number of countries (Agid et al., 2013; Keck et al., 2000; Mallinckrodt et al., 2011; Yildiz et al., 2011), region (Mallinckrodt et al., 2010), number of regions, mean change score on the YMRS in the treatment arm (Agid et al., 2013; Cohen et al., 2010; Gispen-de Wied et al., 2012; Yildiz et al., 2011), and proportion of patients assigned to receive placebo (Kemp et al., 2010; Mallinckrodt et al., 2011; Mallinckrodt et al., 2010; Sysko and Walsh, 2007; Yildiz et al., 2011). Region was classified into three areas: Europe, USA, and Other.
Rating depression over brief time intervals with the Hamilton Depression Rating Scale: Standard vs. abbreviated scales
2015, Journal of Psychiatric ResearchCitation Excerpt :Because these approaches led to different subscales of the HDRS, additional studies examined how well various subscales performed compared to total HDRS score. While some studies showed that shorter subscales improved the rate of response to the outcome measure (e.g. Bech et al., 2010; Faries et al., 2000; Mallinckrodt et al., 2011; Revicki et al., 2010; Santen et al., 2009; Silverstone et al., 2002), others found no noticeable difference (e.g. Ballesteros et al., 2007; McIntyre et al., 2005; Revicki et al., 2010; Ruhe et al., 2005). Boessen et al. (2013) pointed out that some of the differences across studies were likely due to the type of studies used to evaluate the scales.
Effectiveness and acceptability of deep brain stimulation (DBS) of the subgenual cingulate cortex for treatment-resistant depression: A systematic review and exploratory meta-analysis
2014, Journal of Affective DisordersCitation Excerpt :First, the included studies enrolled relatively small numbers of depressed subjects. Second, because pre–post designs are limited with regard to their ability to show causality, we cannot rule out that the clinical improvement observed with DBS was related, for example, to systematic differences in individual patient care, rater bias, placebo effect and/or natural disease course (Mallinckrodt et al., 2011). There is, however, strong indirect evidence suggesting that placebo response rates and spontaneous remission are significantly lower in subjects with TRD as compared to those with uncomplicated MD (Dunner et al., 2006; Fekadu et al., 2009; Fournier et al., 2010).