Introduction

Sequential analysis has been widely researched in medicine, particularly for use in the design and analysis of clinical trials.1 In general, sequential methods involve analysing data as it becomes available, either continuously or in groups. As soon as sufficient information is received, the trial can be terminated.2 The two main types are ‘fully’ (or continuous) sequential and ‘group’ sequential.2 Other types include sequential treatment allocation3 and control charts.4

The earliest known formal application of sequential analysis was the development of control charts in the 1920's for statistical process control.4 Control charts aim to detect unnatural variation caused by some change in a process.5 Data are presented longitudinally on graphs along with lower and upper control limits.5 Types of control charts include Shewhart charts, p-charts, u-charts, and cumulative sum (CUSUM) charts.

Fully sequential analysis, in its modern form, was developed during World War II for quality control in manufacturing.2, 4 In the context of clinical trials, the data are analysed after the inclusion of each patient, or pair of patients if the data are paired.2 The main fully sequential methods are graphical in nature (the sequential probability ratio test and the triangular test, in its original form).2 These tests, in most cases, have expected sample sizes (average stopping times) smaller than the equivalent fixed sample sizes.2, 4, 6

Group sequential procedures were developed in response to alleged shortcomings of fully sequential procedures, such as frequency of analysis.1, 2, 4 In general, analyses are performed either after a certain number of patients are recruited or according to a specific timeframe.2 Common group sequential procedures involve correction of probability (p) values for interim analyses, including the fixed and flexible boundaries approaches, as well as modifications of graphical fully sequential methods.2 Some group sequential procedures also offer expected sample sizes that are smaller than the fixed sample sizes of equivalent nonsequential designs.2, 6

Another form of sequential methodology, somewhat disparate in nature, is sequential treatment allocation. Sequential allocation aims to balance treatment groups in terms of the number of patients as well as any important prognostic factors.1 The simplest version is the so-called biased coin design, where the probability of assignment changes as treatment groups become unbalanced. This concept can be extended to balance treatment groups over multiple stratifying variables.3 These methods of allocation do not necessarily affect the sample size.3 A further class of allocation procedures introduces an adaptive element depending on previous outcomes. Here the objective is to minimise the number of patients receiving a placebo or inferior treatment.1 Conventional methods, on the other hand, require a fixed sample size and, in general, make no formal provisions to allow for early stopping. Stopping a trial early is beneficial to patients receiving a control or inferior treatment, and may also free up resources to be used in other experiments.2, 4, 7, 8, 9 This is particularly important for research involving expensive treatments, and for treatments extending over long periods of time. Clinicians are hence increasingly employing sequential methods in clinical trials.7

In the field of dentistry, clinical trials are characterised by repeated measures over time, often for multiple endpoints.10 Trials frequently proceed for many years, with data collected at annual visits.10 It is of interest to terminate such a trial as early as possible, given ethical concerns for those involved.10, 11, 12 When dentistry trials are stopped, other scientific endeavours may benefit from the freed resources.11 In the hope of attaining these benefits, several researchers have therefore suggested that techniques of sequential analysis be applied in dentistry.10, 11, 12, 13, 14, 15

The aim of this systematic review was to assess and characterise the extent to which sequential analysis has been applied in dentistry and its subfields. This information will be used to provide guidance in the implementation of this useful statistical method for other dental researchers.

Methods

The Cochrane Handbook16 was used as a basic template for the methodology. The types of studies included in this systematic review were clinical trials, excluding review articles and meta-analyses. In particular, trials were restricted to those in the field of dentistry or its subfields, and which applied sequential methods. There were no restrictions on the types of participants, interventions, and outcome measures.

We searched PubMed (since 1966), Web of Science (since 1900), the Cochrane Central Register of Controlled Trials (CENTRAL) and Scopus for articles published up until January 2008. We chose not to search Medline and Embase as it is claimed that they are both covered by Scopus (www.info.scopus.com/detail/what/julie_arnheim.asp). The search terms for PubMed, Web of Science, CENTRAL and Scopus are summarised in Table 1 (where they are partitioned into three columns for clarity). The code for this search is provided in the Addendum. A secondary search of trial references was conducted in Scopus. As PubMed, Web of Science and CENTRAL lack the capacity to search article references, they were not utilised at this stage. Besides the original search terms, this further search was extended to include the names of the main contributors to sequential methods in medicine. They are listed in Table 2. Code for this search is also provided in the Addendum (available on the journal's website www.nature.com/ebd/). The computing limitations of the Scopus search engine required splitting the search into seven smaller searches. Each contained several author names or title flag words, as well as all the dentistry and clinical trial terms.

Table 1 Search terms: primary search
Table 2 Search terms: secondary search

There were no constraints on dates, other than the limitations of the search engines themselves. There were also no language restrictions.

Two authors (PB and GH) examined the article abstracts independently to determine which ones appeared to meet the inclusion criteria. Articles were selected for review if they were clinical trials in the field of dentistry, and returned one or more of the sequential analysis keywords previously mentioned. Disagreements were resolved by discussion and, if necessary, retrieval of the full article. Finally, the selected articles were accessed online, or in original print form where necessary. Thereafter the trials were assessed based on the following criteria: experimental design, whether or not the trial was randomised, the type of sequential analysis used, and treatment type. The corresponding speciality of dentistry was also noted for each trial.

Results

The primary search (search terms in Table 1) yielded 96 articles, 48 of which were unique. The overlaps were eight articles appearing in two databases, 14 appearing in three databases, and four that appeared in all four databases. The secondary, or references search, yielded 131 articles. Scopus lacked the computing capabilities to determine the number of duplicates within the seven smaller searches. The large, albeit minimised, number of search terms overwhelmed the server causing the search engine to time-out. After eliminating articles that were duplicates or not clinical trials, we had 53 unique articles. By retrieving and examining each trial, it was ascertained through discussion that there were a number of false positives. The total number of relevant clinical trials was thereafter narrowed to 18. A flow diagram illustrating the selection process is found in Figure 1.

Figure 1
figure 1

Results: selection flow diagram

The characteristics of the selected trials are summarised below (see also Table 3). In terms of experimental design, the majority17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29 were conventional parallel trials with either two or three treatment arms. In addition, there were some30, 31, 32 split-mouth trials. The other33, 34 trials employed crossover designs. All trials except for three23, 27, 32 stated that some form of randomisation was employed.

Table 3 Results: trial characteristics

The prevalent form of analysis was graphical, with eight19, 25, 26, 30, 31, 32, 33, 34 trials employing some sort of sequential chart. Most19, 25, 30, 31, 32, 33, 34 charts involved plotting a sample path until some rejection or nonrejection boundary was crossed. Four25, 30, 33, 34 trials used Armitage's method. Other methods used included Wald's,31 the triangular test19 and the CUSUM.26 One32 trial did not specify the methods used to produce its charts; another27 used the CUSUM but no graphs were presented. Several trials17, 20, 21, 23, 29 employed sequential treatment allocation. Others18, 22, 24, 28 used p value corrections for interim analyses. The trials employing fully or continuous sequential analyses25, 30, 31, 32, 33, 34 were noted.

The sequential clinical trials employed a variety of dental treatments, including dental restorations,18, 24, 30 dentrifices,23 drugs,25, 28, 32, 34 flap reflection during surgery,31 intra-appointment dressings,20, 29 mechanical and electrical stimuli,26 oral rinses,21, 33 orthodontic appliances,22 oxygen therapy,19 scaling and root planing27 and verbal advice.17 The particular speciality of dentistry corresponding to each trial was also noted. The trials corresponded to endodontics,20, 29 general dentistry,23, 30 oral health,17, 21, 28, 33 oral surgery,19, 25, 31, 32 orthodontics,22 paediatric dentistry, 18, 34 periodontics26, 27 and prosthodontics.24

Discussion

This systematic review was undertaken to establish and characterise the extent to which sequential analysis has been applied in dentistry, and to provide guidance for use in future dental research.

Present results

In a thorough search of PubMed, Web of Science, CENTRAL and Scopus, we found 18 dental clinical trials that applied sequential methods. In light of the extensive theoretical considerations of sequential analysis and the vast number of clinical trials in dentistry, the number of sequential dental trials seems relatively small. The reader should note that this review may not have identified all relevant studies, however, since the search terms might not have been all-encompassing; there may be published trials whose authors applied sequential analysis but did not disclose that this was the case; and finally, there may be unpublished trials which used sequential analysis. The authors acknowledge that sequential methods may not always be appropriate for all dental trials.

There were eight trials19, 25, 26, 30, 31, 32, 33, 34 employing graphical methods. The procedures were successful in all cases, with no trials discontinued for lack of conclusiveness. One trial,32 however, cited the nature of the charts as a limitation. The authors lamented the lack of a quantitative measure of differences in treatment effect. These graphical trials included all four of the oral surgery trials, as well as one trial each from general dentistry, oral health, paediatric dentistry and periodontics. All but one32 mentioned randomisation.

The other trials in this review employed p-value-corrected interim analyses (four trials),18, 22, 24, 28 sequential treatment allocation (five trials)17, 20, 21, 23, 29 or CUSUM tests27. Interim analyses, with p value corrections for multiple looks, are a group sequential method. The numbers of analyses, including interim and final, were two,24, three,28 five22 and seven.18 These trials seemed to employ the group sequential methods for correctness of statistical analysis as opposed to the possibilities of early stopping. Their corresponding subfields of dentistry were oral health, orthodontics, paediatric dentistry, and prosthodontics. All were randomised.

Sequential treatment allocation facilitates balance across treatments and prognostic factors. Three studies17, 23, 29 used a version of Pocock's minimisation method. The other two20, 21 used a two-stage adaptive design. Two of the studies were from endodontics, one from general dentistry, and two from oral health. All but one23 mentioned some form of randomisation.

The last trial27 applied the CUSUM methodology in a nongraphical form. The authors made no mention of randomisation. The trial was from the speciality of periodontics.

A note here is required concerning the two general types of sequential analysis, ie, fully and group sequential. It was found that, of the trials that used sequential analysis, six25, 30, 31, 32, 33, 34 were fully sequential and five18, 19, 22, 24, 28 were group sequential. The fully sequential trials were all graphical. All but one32 claimed to use some form of randomisation. That same trial noted that the lack of quantitative estimates was a potential limitation of the trial design. Three quarters of the trials involving drugs and three quarters of the oral surgery trials were fully sequential. By contrast, the group sequential trials were distributed over various areas of dentistry. The diminished potential for early stopping under group sequential trials did not seem to be a hindrance here. Indeed, most clinicians seemed intent on letting their trials run to the fullest extent.

There were also two trials35, 36 that explicitly chose not to implement sequential methodology in their trial designs. One35 involved a treatment period of 4 years and a discontinuation period of 5 years thereafter. The authors performed an interim analysis after the first 4 years, and a final analysis after 9 years had elapsed. They claimed that the second, or final, measurement was the only one that effectively measured their primary endpoint but noted that their conclusions would have been unchanged by adjusting significance levels. The other study36 included a halfway interim analysis as well as a final analysis. These authors wrote that the corrected boundaries would not have been much less than 0.05, but they did not calculate them explicitly. It was noted, however, that their reported p values were either very small (P<0.001) or very large (P>0.15) so that the adjustment would probably not have altered their conclusions. These trials were from general dentistry (use of sealants and varnish)35 and orthodontics (use of toothbrushes).36

Theoretical papers

The searches of PubMed, Web of Science, CENTRAL and Scopus yielded some relevant theoretical papers10, 11, 12, 13, 14, 15 which are briefly reviewed here.

In an early treatise, Smith and O'Mullane12 proposed the use of a two-sample sequential t test for trials of caries prophylactic agents. The t test was illustrated with an example using annual incremental data. Armitage et al.11 developed an alternative repeated significance test for longitudinal data with comparisons between interventions. The test was illustrated with reference to a trial comparing the efficacy of two types of fluoride dentrifice on dental caries.

In a general survey of the design and analysis of periodontal clinical trials, Imrey and Chilton13 noted that interim analyses should be “undertaken systematically and... compensated for by the method of establishing statistical significance at each look.” Van der Glas and colleagues15 proposed the use of several CUSUM methods in reflex studies, illustrated by teeth stimulation data. Petrie et al.14 noted that sequential methods can be applied in dentistry, both group and fully sequential, but no specific examples were given. Leroux et al.10 further advocated the use of sequential methods for longitudinal clinical trials in dentistry. They developed a sequential test for trials with multiple endpoints and repeated measures over time. In particular, they noted its applicability to trials for caries prevention, and illustrated its use in a trial to assess the safety of dental amalgam fillings.

General discussion

There are a number of advantages and disadvantages in using fully and group sequential analysis. The main benefits are ethical and economic, arising from the possibility of early stopping. As previously noted, stopping a trial early may benefit those participants receiving inferior treatments and might free up resources to be used elsewhere.2, 4, 7, 8, 9 Further, through early stopping under the null hypothesis, clinicians may avoid unknown side-effects.2 In general, early stopping might engender substantial sample size reductions:2, 4 ie, the expected sample size may be smaller than the sample size of a nonsequential design. Lastly, ignoring the inherently sequential nature of a clinical trial can incur bias.9

There are some disadvantages. For both group and fully sequential analyses, the data must be timely and highly accurate.7 There may be significant costs incurred by this right-in-time organisation and statistical analyses.9 Indeed, the logistics are more complicated than for conventional trials.2, 9 It is possible that the length of a sequential trial may be longer than its single-stage counterpart.9 Also, the maximum sample size (truncation point) may be larger6, 37 than the sample size for a nonsequential trial, and in some cases infinite.2 Reductions in patient numbers only occur if the endpoint is available quickly enough relative to the patient recruitment rate.2, 37 Indeed, an extreme situation would have all patients included prior to the first possible data analysis.2 Sometimes it might not be unethical to continue a trial to estimate a treatment effect or to achieve higher power.2 Trials that stop early are prone to exaggerating treatment effects8 and stopping early also diminishes the probability of uncovering adverse effects so that evidence of harm does not emerge. Although the disadvantages are not trivial, Wegscheider9 suggests that the ethical advantages dominate.

Sequential allocation is an alternative to complete randomisation, where all treatment assignments are equally likely and block-randomised designs where patients are assigned to treatments within groups (eg, male and female).38 The main advantage of sequential allocation is that allocations are balanced across treatments and prognostic factors. In addition, decreases in selection bias and susceptibility to imbalance over permuted block designs have been observed.3 Some authors have suggested that it is ill-informed not to incorporate accruing information into the trial design.39 Although the method of sequential allocation and its adaptive counterparts is not new, the techniques have seldom been used in practice.39 According to Halpern and Brown,3 these sequential designs may be expensive and difficult to implement. They noted that the gains in balance and efficiency are negligible relative to complete randomisation. The reader may recall that randomisation is often said to validate statistical tests.40 In the case of sequential (nonrandom) allocation, conventional statistical analyses are not valid.3 These problems are exacerbated by a lack of clear examples in the literature.39 We emphasise that nonrandom allocation methods are, in general, not advisable because they may lead to overestimated treatment effects.41 Hence, the sequential allocation technique should be applied with due caution.

There is debate over whether it is best to use fully sequential or group sequential procedures. Fully sequential procedures generally offer larger expected sample size8 and time reductions.9 Further, fewer patients are exposed to the poorer treatment.8 On the other hand, group sequential methods are much simpler2, 7 and more flexible2 than their continuous counterparts. Group sequential trials are shortened less often and are less biased.9 They are desirable when the need arises to estimate treatment effects and also to demonstrate superiority.9 Both types have some shortcomings, however. Fully sequential analysis is rarely applied in medicine because of the frequency of data analysis.2 If the data monitoring committee is unable to meet at short notice, accrual continues. This is inefficient compared with group sequential methods.37 There are also a number of statistical issues in applying fully sequential analysis. First, conventional point estimates and p values are incorrect.2, 7, 9 Secondly, adjustment of the final analysis is more marked than for group sequential procedures.8 Lastly, there is the possibility of incurring bias when pooling fully sequential trials with others for meta-analysis.9 Group sequential methods, in contrast, delay the potential for early stopping.7 Also, any unplanned interim analyses, especially those that are data-provoked, could cause data interpretation problems and exaggerate treatment effects.2

The debate is far from resolved. Any decision to choose one over the other must come with careful consideration of costs and benefits. The solution may lie in new procedures such as that outlined by Gombay and Hussein.6 They proposed a sequential t test with no restrictions on the timing and number of interim analyses. Indeed, the test enables clinicians to switch from periodic interim analyses to continuous monitoring.6

A simulation study of adaptive designs demonstrated relative efficiencies of 100–120% versus nonadaptive alternatives.39 This corresponds to sample sizes for the adaptive designs that are 83–100% of those of the fixed sample size designs.39 In some clinical trials, it might be useful for the investigators to set the maximal sample size (truncation point) and design sequential procedures to achieve the best power and average stopping time. In group and fully sequential procedures, the maximum sample size is about 105–130% of the fixed sample size and the average stopping time is about 70–80% of the fixed sample size in the case of independent two sample t tests.4, 6 Recall that the sample size depends on the power, the significance level and the parameters specified under the alternative hypothesis. In group sequential procedures, the number of interim analyses must be set before the trial begins. The maximum sample size and the expected sample size depend also upon the number of interim analyses. One might argue, then, that the possible increase in the maximum sample size is negligible relative to the benefits, when group and fully sequential procedures are applied.

Finally, we consider another version of sequential analysis, which has been recently proposed. The Pharmaceutical Research and Manufacturers of America Adaptive Design Working Group, established in 2005, broadly defines an adaptive design as one that, “... uses accumulating data to decide on how to modify aspects of the study without undermining the validity and integrity of the trial.”42 According to the working group, the most desirable sequential design should be flexible to: (1) allocate the subjects to available arms; (2) re-assess the sample size; (3) determine when to stop the trial for efficacy, harm and futility; and (4) make the interim and terminal decisions.42 The adaptive design is a generalisation of group sequential methods. It may be the most practical sequential technique for clinicians in dentistry, because it permits redesigning of the next stage with emphasis on one or more of the aforementioned rules, (1)-(4). The interested reader is encouraged to seek out the article by Bauer and Brannath.43

In summary, this systematic review has examined the use of sequential analysis in dentistry as well as its general advantages and disadvantages. We found 18 clinical trials which applied sequential methodology.

Conclusions

  • The small number of relevant trials in this review might suggest that sequential analysis is underused in dental fields and its subfields.

  • Theoretical and methodological resources are already in place to support future applications of sequential analysis in dentistry.

  • The direction of future research may lie in the flexible group sequential and adaptive designs mentioned above.