Abstract
Background: There are limited published comparative data regarding the sensitivity and timing of early signal detection with commonly used signal detection methods (SDMs), including the reporting odds ratio (ROR), proportional reporting ratio (PRR), information component (IC) and gamma Poisson shrinker (GPS).
Objective: To examine the sensitivity and timing of early signal detection across four SDMs using the Adverse Events Reporting System (AERS) database of the US Food and Drug Administration.
Methods: The four SDMs were applied to retrospectively detect ten confirmed drug-event combinations (DECs). The sensitivity to detect adverse events was defined as the percentage of DECs detected by the respective SDMs as positive signals. The timing of early signal detection was measured by comparing the index date of withdrawal (IDW), defined as the date on which the drug was removed from the market, with the index date of detection (IDD), defined as a date on which the signal was significantly detected by the SDM.
Results: The estimated sensitivity was 100% for ROR, 90% for PRR and IC and 70% for GPS. The sensitivity increased with increasing numbers of reports per DEC. Compared with the IDW, the signals were detected on average 10 quarters earlier by ROR, 9 quarters earlier by PRR, 9.9 quarters earlier by the IC and 4.7 quarters earlier by GPS.
Conclusions: The sensitivity and timing of early signal detection varies across the four SDMs. Numerically, the ROR showed better performance in sensitivity and early signal detection based on ten selected DECs. Given the limited number and range of DECs selected in this study and the unavailability of specificity assessment, further large-scale prospective studies are warranted in order to provide better guidance on the selection of SDMs.
This is a preview of subscription content, log in via an institution to check access.
References
Hauben M, Reich L. Safety related drug-labeling changes: findings from two data mining algorithms. Drug Saf 2004; 27 (10): 735–44
Rossi AC, Knapp DE, Anello C, et al. Discovery of adverse drug reactions: a comparison of selected phase studies with spontaneous reporting methods. JAMA 1983; 249 (16): 226–8
van Puijenbroek EP, Diemont WL, Grootheest K. Application of quantitative signal detection in the Dutch Spontaneous Reporting System for adverse drug reactions. Drug Saf 2003; 26 (5): 293–301
Evans SJV, Waller PC, Davis S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf 2001; 10 (6): 483–6
DuMouchel W. Bayesian data mining in large frequency tables, with an application to FDA spontaneous reporting system. Am Stat 1999; 53 (3): 177–90
DuMouchel W, Smith ET, Beasley R, et al. Association of asthma therapy and Churg-Strauss Syndrome: an analysis of postmarketing surveillance data. Clin Ther 2004; 26 (7): 1092–104
Bate A, Lindquist M, Edwards IR, et al. A Bayesian neural network method for adverse drug reaction signal generation. Eur J Clin Pharmacol 1998; 54 (4): 315–21
Bate A, Lindquist M, Edwards IR, et al. A data mining approach for signal detection and analysis. Drug Saf 2002; 25 (6): 393–7
van Puijenbroek EP, Bate A, Leufkens HGM, et al. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiol Drug Saf 2002; 11 (1): 3–10
Kubota K, Koide D, Hirai T. Comparison of data mining methodologies using Japanese spontaneous reports. Pharmacoepidemiol Drug Saf 2004; 13 (6): 387–94
Waller P, van Puijenbroek EP, Egberts A, et al. The reporting odds ratio versus the proportional reporting ratio: ‘deuce’. Pharmacoepidemiol Drug Saf 2004; 13 (8): 525–6
Almenoff JS, LaCroix KL, Yuen NA, et al. Comparative performance of two quantitative safety signaling methods: implications for use in a pharmacovigilance department. Drug Saf 2006; 29 (10): 875–87
Medical Dictionary for Regulartory Activities (MedDRA) [online]. Available from URL: http://www.fda.gov/medwatch/REPORT/meddra.htm[Accessed 2006 Nov]
Medical Dictionary for Regulatory Activities Maintenance and Support Services Organization (MSSO) [online]. Available from URL: http://www.meddramsso.com/MSSOWeb/index.htm [Accessed 2006 Nov]
Li CJ, Xia JL, Deng JX, et al. A comparison of measures of disproportionality for signal detection on adverse drug reaction spontaneous reporting database of Guangdong province in China. Pharmacoepidemiol Drug Saf. Epub 2008 Apr 23
Almenoff JS, Tonning JM, Gould AL, et al. Perspectives on the use of data mining in pharmacovigilance. Drug Saf 2005; 28 (11): 981–1007
Council for International Organizations of Medical Sciences. Introductory guide for standardized MedDRA queries (SMQs) Version 11.0. Reston (VA): Council for International Organizations of Medical Sciences (CIOMS)/MedDRA [Medical Dictionary for Regulatory Activities] Maintenance and Support Services Organization (MSSO), 2008 Mar. MSSO-DI-6226-11.0.0
Acknowledgements
No sources of funding were used to assist in the preparation of this study. The authors have no conflicts of interest that are directly relevant to the content of this study. The opinions and conclusions expressed in this manuscript are solely those of authors.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Chen, Y., Guo, J.J., Steinbuch, M. et al. Comparison of Sensitivity and Timing of Early Signal Detection of Four Frequently Used Signal Detection Methods. Pharm Med 22, 359–365 (2008). https://doi.org/10.1007/BF03256733
Published:
Issue Date:
DOI: https://doi.org/10.1007/BF03256733