A systematic review found that deviations from intention-to-treat are common in randomized trials and systematic reviews

Abstract

Objectives

To describe the characteristics, and estimate the incidence, of trials included in systematic reviews deviating from the intention-to-treat (ITT) principle.

Study Design and Setting

A 5% random sample of reviews were selected (Medline 2006–2010). Trials from reviews were classified based on the ITT: (1) ITT trials (trials reporting standard ITT analyses); (2) modified ITT (mITT) trials (modified ITT; trials deviating from standard ITT); or (3) no ITT trials.

Results

Of 222 reviews, 81 (36%) included at least one mITT trial. Reviews with mITT trials were more likely to contain trials that used placebo, that investigated drugs, and that reported favorable results. The incidence of reviews with mITT trial ranged from 29% (17/58) to 48% (23/48). Of the 2,349 trials, 597 (25.4%) were classified as ITT trials, 323 (13.8%) as mITT trials, and 1,429 (60.8%) as no ITT trials. The mITT trials were more likely to have reported exclusions compared to studies classified as ITT trials and to have received funding.

Conclusion

The reporting of the type of ITT may differ according to the clinical area and the type of intervention. Deviation from ITT in randomized controlled trials is a widespread phenomenon that significantly affects systematic reviews.

Introduction

Intention-to-treat (ITT) is the recommended standard approach to analyze data from randomized controlled trials. This method requires that patients are analyzed according to their original random allocation to preserve the prognostic balance, thereby minimizing selection bias and confounding [1], [2], [3].

Trials that use a so-called “modified ITT” (mITT) approach are appearing in increasing numbers in the medical literature. The features of the mITT analysis are highly variable, with authors often using more than one criterion to describe this alternative approach [4]. For example, in a trial that evaluated the efficacy of topiramate, compared to placebo, to reduce the mean percentage change in body weight, the authors stated that their mITT population comprised “randomized individuals who had at least one dose of study medication, at least one post-baseline efficacy assessment, and the opportunity to complete at least 44 weeks of medication.” Despite the use of “mITT,” the analysis performed was substantially a “per-protocol (PP) analysis” because the real reason for excluding 48% of the participants was withdrawal [5]. In a trial where rofecoxib and celecoxib were compared to acetaminophen, to reduce pain in patients with osteoarthritis, the authors declared that the efficacy analysis was “conducted using a modified intent-to-treat approach, whereby all patients who took at least 1 dose of study medication were included in the analysis” [6]. Carefully assessing this last study, there is no apparent patient exclusion from analysis, and the deviation from the ITT remains unclear.

The frequency of trials using an mITT approach (475 trials, mostly published between 2000 and 2006) may well have been underestimated because the inquiry was limited to the trials that explicitly used the word “modified” in the description of the deviation from the ITT analysis [4]. Indeed, in a subsequent study [7], a new selection criterion was used which also included studies that deviated from an ITT analysis without specifically using the word “modified” (e.g., a study that compared cyclooxygenase-2 inhibitors with acetaminophen declared its ITT population as “all patients who received at least 1 dose of assigned study medication”). This generated a higher than expected number of trials (32 trials that deviated from the ITT analysis, without using the word “modified” in addition to 24 trials that used the word “modified”), underlining that the trials that deviated from the ITT analysis should, to a large extent, be considered as mITT reporting trials. In that cross-sectional study [7], trials classified as mITT trials were more likely to report post-randomization exclusions and to have received funding from for-profit enterprises than trials that reported standard ITT. This analysis was, however, limited to trials published in three general and three specialty journals that were more likely to publish trials with mITT approach. Thus, there is a need to understand how widespread the phenomenon of the deviation from ITT is in the medical literature especially within systematic reviews.

Systematic reviews are generally considered the best source of evidence for clinical decision making and are often used as a baseline tool for guideline developers. To be a reliable tool, reviews need to be well conducted, and any bias present in the included trials should be made explicit, in order not to influence the conclusions of the review. Hence, readers of systematic reviews should be aware of the characteristics and reporting of systematic reviews including the type and characteristics of the trials included in these reviews [8]. In a meta-epidemiological evaluation [9], we have shown that the reported increase in frequency of the trials that deviated from ITT, in the medical literature [4], influenced the estimate of the treatment effect reported in systematic reviews and meta-analyses. Using the initial sample of reviews for which we performed the meta-epidemiological evaluation [9], we performed a further investigation regarding the incidence of trials that deviated from ITT in systematic reviews, as well as how these reviews differ in terms of several characteristics and reporting from reviews that do not contain trials reported deviation from ITT.

The primary aim of this study was to assess the epidemiology of ITT reporting of randomized trials included in systematic reviews; to assess the incidence of trials that deviated from ITT in meta-analyses; to compare the difference in characteristics between systematic reviews, with at least one randomized trial that deviated from ITT included in meta-analyses, with systematic reviews with randomized trials that did not deviate; and to compare the difference in characteristics among trials based on ITT reporting.

We believe that the results of the present study will be important for various professionals engaged in the design, conduct, and reporting of clinical trials and meta-analyses, as well as for those involved in guideline development and clinical decision making. Knowledge of potential bias associated with the type of ITT reporting, and of the incidence of trials that deviated from an ITT analysis, shall prompt clinical researchers to adopt adequate randomization modalities, minimize attrition bias, as well as use adequate methods of data analysis and stimulate clinicians to carefully consider whether to adopt interventions supported solely by studies reporting a modified ITT (mITT) approach.