ReviewA systematic review found that deviations from intention-to-treat are common in randomized trials and systematic reviews
Introduction
Intention-to-treat (ITT) is the recommended standard approach to analyze data from randomized controlled trials. This method requires that patients are analyzed according to their original random allocation to preserve the prognostic balance, thereby minimizing selection bias and confounding [1], [2], [3].
Trials that use a so-called “modified ITT” (mITT) approach are appearing in increasing numbers in the medical literature. The features of the mITT analysis are highly variable, with authors often using more than one criterion to describe this alternative approach [4]. For example, in a trial that evaluated the efficacy of topiramate, compared to placebo, to reduce the mean percentage change in body weight, the authors stated that their mITT population comprised “randomized individuals who had at least one dose of study medication, at least one post-baseline efficacy assessment, and the opportunity to complete at least 44 weeks of medication.” Despite the use of “mITT,” the analysis performed was substantially a “per-protocol (PP) analysis” because the real reason for excluding 48% of the participants was withdrawal [5]. In a trial where rofecoxib and celecoxib were compared to acetaminophen, to reduce pain in patients with osteoarthritis, the authors declared that the efficacy analysis was “conducted using a modified intent-to-treat approach, whereby all patients who took at least 1 dose of study medication were included in the analysis” [6]. Carefully assessing this last study, there is no apparent patient exclusion from analysis, and the deviation from the ITT remains unclear.
The frequency of trials using an mITT approach (475 trials, mostly published between 2000 and 2006) may well have been underestimated because the inquiry was limited to the trials that explicitly used the word “modified” in the description of the deviation from the ITT analysis [4]. Indeed, in a subsequent study [7], a new selection criterion was used which also included studies that deviated from an ITT analysis without specifically using the word “modified” (e.g., a study that compared cyclooxygenase-2 inhibitors with acetaminophen declared its ITT population as “all patients who received at least 1 dose of assigned study medication”). This generated a higher than expected number of trials (32 trials that deviated from the ITT analysis, without using the word “modified” in addition to 24 trials that used the word “modified”), underlining that the trials that deviated from the ITT analysis should, to a large extent, be considered as mITT reporting trials. In that cross-sectional study [7], trials classified as mITT trials were more likely to report post-randomization exclusions and to have received funding from for-profit enterprises than trials that reported standard ITT. This analysis was, however, limited to trials published in three general and three specialty journals that were more likely to publish trials with mITT approach. Thus, there is a need to understand how widespread the phenomenon of the deviation from ITT is in the medical literature especially within systematic reviews.
Systematic reviews are generally considered the best source of evidence for clinical decision making and are often used as a baseline tool for guideline developers. To be a reliable tool, reviews need to be well conducted, and any bias present in the included trials should be made explicit, in order not to influence the conclusions of the review. Hence, readers of systematic reviews should be aware of the characteristics and reporting of systematic reviews including the type and characteristics of the trials included in these reviews [8]. In a meta-epidemiological evaluation [9], we have shown that the reported increase in frequency of the trials that deviated from ITT, in the medical literature [4], influenced the estimate of the treatment effect reported in systematic reviews and meta-analyses. Using the initial sample of reviews for which we performed the meta-epidemiological evaluation [9], we performed a further investigation regarding the incidence of trials that deviated from ITT in systematic reviews, as well as how these reviews differ in terms of several characteristics and reporting from reviews that do not contain trials reported deviation from ITT.
The primary aim of this study was to assess the epidemiology of ITT reporting of randomized trials included in systematic reviews; to assess the incidence of trials that deviated from ITT in meta-analyses; to compare the difference in characteristics between systematic reviews, with at least one randomized trial that deviated from ITT included in meta-analyses, with systematic reviews with randomized trials that did not deviate; and to compare the difference in characteristics among trials based on ITT reporting.
We believe that the results of the present study will be important for various professionals engaged in the design, conduct, and reporting of clinical trials and meta-analyses, as well as for those involved in guideline development and clinical decision making. Knowledge of potential bias associated with the type of ITT reporting, and of the incidence of trials that deviated from an ITT analysis, shall prompt clinical researchers to adopt adequate randomization modalities, minimize attrition bias, as well as use adequate methods of data analysis and stimulate clinicians to carefully consider whether to adopt interventions supported solely by studies reporting a modified ITT (mITT) approach.
Section snippets
Data source and search strategy
Our articles of interest were systematic reviews that were published between 2006 and 2010. To identify the reviews, we launched Montori's highly specific search strategy in PubMed (MEDLINE [Title/Abstract] OR (systematic[Title/Abstract] AND review[Title/Abstract] OR meta-analysis[Publication Type])) [10]. Subsequently, we randomly selected 5% of the records, from each year, for abstract screening.
Selection of systematic reviews or meta-analyses
We were interested only in reviews of therapeutic or preventive interventions that included at
Results
Our electronic search in Medline identified 44,900 reviews published between 2006 and 2010 from which 2,245 abstracts (5%) were randomly selected and screened. We excluded 1,830 records based on the title and abstract. The resulting 415 full-text reviews were evaluated and, after excluding 193 articles with reasons, 222 reviews remained for further assessment. Of the 2,701 potentially eligible trials that were included in the 222 reviews, we were unable to obtain 26 full texts (1%), and 326
Summary of findings
After randomly selecting 5% of the intervention reviews published in Medline, we compared the characteristics of systematic reviews according to the type of ITT reported in the included randomized trials. We found that there was a substantial difference between the two groups of reviews (reviews with mITT trials and reviews without mITT trials). The clinical areas of most relevance for reviews with mITT trials were neuropsychiatric and rheumatic diseases, whereas gastrointestinal and
Conclusions
In this 5-year sample of reviews, we showed that the reporting of the type of ITT may differ according to the clinical area and the type of intervention. The use of an mITT approach, or deviation from ITT, in randomized controlled trials, is a widespread phenomenon that significantly affects systematic reviews. Such trials are specific to new experimental drugs, are characterized by post-randomization exclusions and significant results and are dominated by industrial funding.
Acknowledgments
The authors thank Joseph M. Rimland for editing and commenting the article.
Authors' contributions: I.A. and A.M. conceived and designed the study; I.A., R.D.F., M.L.L., M.O., F.C., and A.G. were involved in the data acquisition; I.A., A.M., A.I., M.M., G.L., and P.E. analyzed and interpreted the data; and I.A., A.M., R.D.F., M.L.L., M.O., F.C., A.G., A.I., M.M., G.L., and P.E. contributed in the drafting and revising the study and have approved submission of the final version of the article.
References (30)
- et al.
Discordance between reported intention-to-treat and per protocol analyses
J Clin Epidemiol
(2007) - et al.
A systematic review finds variable use of the intention-to-treat principle in musculoskeletal randomized controlled trials with missing data
J Clin Epidemiol
(2015) - et al.
Modified intention-to-treat analysis did not bias trial results
J Clin Epidemiol
(2016) Intention-to-treat analysis: implications for quantitative and qualitative research
Int J Epidemiol
(1992)- et al.
Post-randomisation exclusions: the intention to treat principle and excluding patients from analysis
BMJ
(2002) - et al.
Intention to treat—who should use ITT?
Br J Cancer
(1993) - et al.
Modified intention to treat reporting in randomised controlled trials: a systematic review
BMJ
(2010) - et al.
Topiramate: long-term maintenance of weight loss induced by a low-calorie diet in obese subjects
Obes Res
(2004) - et al.
Efficacy of rofecoxib, celecoxib, and acetaminophen in osteoarthritis of the knee: a randomized trial
JAMA
(2002) - et al.
Modified versus standard intention-to-treat reporting: are there differences in methodological quality, sponsorship, and findings in randomized trials? A cross-sectional study
Trials
(2011)
Epidemiology and reporting characteristics of systematic reviews
PLoS Med
Deviation from intention to treat analysis in randomised trials and treatment effect estimates: meta-epidemiological study
BMJ
Optimal search strategies for retrieving systematic reviews from Medline: analytical survey
BMJ
The intention-to-treat approach in randomized controlled trials: are authors saying what they do and doing what they say?
Clin Trials
What is meant by intention to treat analysis? Survey of published randomised controlled trials
BMJ
Cited by (34)
Behavioral and Social Research to Accelerate the Geroscience Translation Agenda
2020, Ageing Research ReviewsCitation Excerpt :By the end of a randomized trial, this situation may re-introduce the bias and confounding that random assignment to trial arms was intended to eliminate (Demets and Cook, 2019). Adherence confounding can be avoided by intention-to-treat (ITT), but unfortunately only one-quarter of trials follow ITT, according to a survey of 2,349 trials (Abraha et al. 2017). For this reason, the marked heterogeneity in causal influences on humans’ aging will probably complicate and even compromise clinical trials of anti-aging therapies.
Double-blind evaluation of homeopathy on cocaine craving: a randomized controlled pilot study
2018, Journal of Integrative MedicineCitation Excerpt :She was responsible for giving the study medication to each patient and had, therefore, access to the content of the envelopes (group 1 or group 2). Modified intent-to-treat analysis comprised all available cases [23], i.e., all randomized subjects who had at least one post-baseline effectiveness assessment. Missing data were imputed by carrying the last observation forward.
Clinical studies in restorative dentistry: Design, conduct, analysis
2018, Dental MaterialsCitation Excerpt :These are seldom missing completely at-random; only in this case, an available case analysis is suitable, where lost patients are either not at all entered into the analysis or handled as censored [6]. It is far more likely that drop-outs do not occur at-random: For example, patients with negative outcomes (being discouraged or in urgent need of help, seeking it elsewhere) or positive outcomes (perceiving no need to re-attend) might over-proportionally drop out of a trial, often with significant impact on a studies’ conclusion [78,79]. If sufficient data are available to test if drop-outs were associated with certain (known) parameters (age, gender, etc.), then it may be possible to adjust outcomes accordingly.
Non-pharmacological interventions to prevent falls in older patients: Clinical practice recommendations – the SENATOR ONTOP Series
2017, European Geriatric MedicineCitation Excerpt :GRADE is a systematic, standardized method to evaluate the quality of evidence. The quality assessment is based on risk of bias of the study (including allocation concealment, blinding of outcome assessor [28], incomplete outcome data [29,30] and selective outcome reporting [31,32]), the consistency, the directness in the applicability of the evidence, the precision of the treatment effect estimate and publication bias. If there are limitations in any of the aforementioned categories, the quality of the evidence can be downgraded by 1 or 2 levels.
Funding: The study was funded by the Ministry of Health, Italy (RF-2009-1549561). The study sponsor had no role in the study design; collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.
Conflict of interest: None.