Assessment of therapeutic safety in systematic reviews: literature review

Methods and results

We performed systematic literature searches to identify all systematic reviews and meta-analyses on the efficacy or effectiveness and the safety of any type of therapeutic intervention. Data sources were Medline (1966-December 2000) and the Cochrane Library (2000, issue 4). The title, abstract, and keywords of papers were searched using the terms efficacy, effectiveness, systematic review, meta-analysis, safety, adverse effect, adverse event, and adverse reaction and their derivatives. No language restrictions were imposed. The title and abstract of all studies were read and data were according to the criteria defined. Studies were included only if they stated that they were systematic reviews or meta-analyses or if they indicated that the available data were systematically searched for, appraised, and summarised. Studies that included no abstract or that provided no data on humans were excluded. Data were assessed according to the year of the study, and the studies were categorised into three categories: systematic reviews on efficacy or effectiveness, systematic reviews that included safety aspects as a secondary outcome measure (safety category A), and systematic reviews with a primary focus on the safety of the therapy (safety category B). Study selection, data extraction, and evaluation were performed independently by the two investigators, and discrepancies were resolved through discussion.

The results (table) showed a continuous increase in the number of systematic reviews and meta-analyses published between 1966 and December 2000. The number of studies that included safety aspects as a secondary outcome measure (category A) amounted to just over one quarter of the total number of studies on efficacy or effectiveness. The number of systematic reviews that assessed safety aspects as the primary focus of the investigation (category B) amounted to only 3-5% (table).

Comment

Systematic reviews and meta-analyses provide little information on the safety aspects of therapeutic interventions. Most systematic reviews include data on safety only in as much as the primary studies, usually randomised clinical trials, report on adverse events. This seems inadequate. Randomised clinical trials typically assess only a small number of patients and thus the chances of detecting rare adverse events are small. Randomised clinical trials are usually of short duration and thus cannot identify delayed adverse events.2 The assessment of safety has to go far beyond randomised clinical trials and should use various methods, including post-marketing surveillance studies, spontaneous reporting schemes, and epidemiological investigations.3 Systematic reviews on the safety of therapeutic interventions should take these considerations into account and should combine data from various types of studies.4 Safety aspects have so far not been adequately investigated by systematic reviews. One challenge of evidence based medicine therefore is to address safety issues more systematically than has been done before.