Statistical Considerations for Performing Multiple Tests in a Single Experiment. 2. Comparisons Among Several Therapies
Section snippets
Overall Preliminary Test.
With use of an overall preliminary test, a null hypothesis is established specifying that no difference exists among any of the groups. If 10 drugs are being studied, for example, a test statistic (referred to as F, analogous to t in the t test for comparing two therapies) is derived by dividing a measure of the variability among the 10 group means by a measure of the variability expected by chance. One obtains the corresponding P value from suitable tables (the larger the F statistic, the
EXAMPLE
We will consider a double-blind crossover study in which nine marketed analgesics and a placebo were evaluated in 57 patients with definite pain problems as a result of unresectable cancer.4 Although all possible pairwise comparisons were of interest, only comparisons with aspirin and placebo were reported. Thus, the total number of comparisons was 17. If no difference in efficacy was detected among any of the preparations, separate paired t tests performed at the 0.05 level would probably show
ALTERNATIVE EXAMPLE
Notice that the purpose of the study in the example determined the precise formulation of the study questions and the corresponding data analysis used to answer them. Suppose, however, that the same study had been performed for a different purpose and had addressed somewhat different questions. Specifically, suppose that the manufacturer of propoxyphene (Darvon) had performed this study and that the specific aims of the study had been to evaluate propoxyphene (1) relative to aspirin, (2)
COMMENTS
At this point, readers may ask which of the many methods of analysis that have been described is the best. Perhaps the most important point to be made is that no one method is always the “best” method.
The questions that a study is intended to answer must be clearly stated beforehand. Failure to consider this basic principle often may lead to an overreliance on per-experiment error rates. As an illustration, suppose one investigator conducts an experiment to answer the following question: “Are
ACKNOWLEDGMENT
We thank Charles G. Moertel, M.D., and his colleagues for the use of their data on pain relievers.
REFERENCES (7)
A multiple comparison procedure for comparing several treatments with a control
J Am Stat Assoc
(1955)New tables for multiple comparisons with a control
Biometrics
(1964)
Cited by (37)
An introduction to the use of interim data analyses in clinical trials
1993, Annals of Emergency MedicineProbucol reduces plasma lipid peroxides in man
1992, AtherosclerosisStatistical concepts and methods for the reader of clinical studies in emergency medicine
1991, Journal of Emergency MedicineStatistical concepts for research in emergency medical services
2021, Emergency Medical Services: Clinical Practice and Systems Oversight: Third Edition
Individual reprints of this article are not available. The entire six-part series will be available for purchase as a bound booklet from the Proceedings Circulation Office in December.