When and how to update systematic reviews
Abstract
This is a protocol for a Cochrane Review (Methodology). The objectives are as follows:
To identify, describe and assess methods addressing:
1) when to update systematic reviews, and
2) how to update systematic reviews.
Background
Introduction
Professionals involved in the delivery of health are faced with the challenge of how best to keep themselves up‐to‐date with advances in their speciality (Cook 1997a; Mulrow 1994). An efficient use of reading time would be to focus on systematic reviews of individual studies (Higgins 2005). Systematic reviews are regularly incorporated into guidelines and appraisal tools highlight them as a core method of evidence synthesis (AGREE 2005; Cook 1997b).
Systematic reviews are less useful if they are not up‐to‐date (Eccles 2001). Some health care interventions currently known to be effective will in future be shown to be ineffective or harmful, or vice‐versa (Chalmers 1994). New interventions or health outcomes will emerge (Shekelle 2001). Ignoring these changes could undermine the validity of systematic reviews and clinical practice guidelines, and systematic reviews therefore need to be periodically updated (Chalmers 1994; Claxton 2005; Higgins 2005).
When to update systematic reviews
Updating a systematic review can be as costly and time‐consuming as conducting the original review or developing the original guideline (Barrowman 2003; Eccles 2001; Stead 2001). Therefore, it is of paramount importance to know whether or not a systematic review needs to be updated. The answer largely depends on the rate of development of a particular health care field. Frequent updating of systematic reviews in slowly developing areas may result in a waste of resources, whereas systematic reviews in rapidly developing areas may quickly become outdated. Arbitrarily chosen fixed updating frequencies (e.g., every two or three years) of systematic reviews or clinical guidelines, as suggested by some authors (Eccles 1996; Higgins 2005; Lancaster 2001), might be an overly simplistic approach for resolving this complex issue. The timing could be influenced by health care field, public health importance, economic implications, and nature of the health related problem. Currently there are no standard guidelines on when to update systematic reviews (Chapman 2002; French 2004; Weller 1998).
How to update systematic reviews
Several authors have suggested how systematic reviews should be updated (Barrowman 2003; Higgins 2005; Ioannidis 1999; Pogue 1997; Stead 2001). In a short review, Weller proposed some strategies for managing and updating the evidence included in systematic reviews and clinical guidelines (Weller 1998). Cumulative meta‐analysis allows the monitoring of the statistical summary estimates over time as results from new studies become available (Lau 1995; Pogue 1997). Barrowman et al. described a technique that can be applied to a meta‐analysis with a statistically non‐significant summary estimate to determine whether new information could potentially overturn the non‐significant result (Barrowman 2003). Chalmers described the methods used for updating reviews in pregnancy and childbirth (Chalmers 1993). The various methods suggested for updating systematic reviews have not yet been reviewed.
Objectives
To identify, describe and assess methods addressing:
1) when to update systematic reviews, and
2) how to update systematic reviews.
Methods
Criteria for considering studies for this review
Types of studies
Empirical studies describing one or more methods for updating systematic reviews in health care.
Types of data
Any human population.
Types of methods
Any types of method.
Types of outcome measures
The following data will be extracted for each method:
-
General description (elements of 'when' and 'how')
-
Underlying assumptions
-
Advantages
-
Disadvantages
-
Time and amount of resources (e.g. the number of authors, scope of search)
-
Change in conclusions of original systematic review after the application of method (if applicable)
-
Comprehensiveness (e.g. consideration of when and how to update, field of health care, statistical calculations, original vs. modified search strategy)
Search methods for identification of studies
The search will cover 1955 to present. There will be no language restriction. To identify relevant reports, the following sources will be searched:
Subject searching
Cochrane Methodology Register (The Cochrane Library)
MEDLINE (Ovid 1966 to present)
PsycInfo (1955 to present) using a modification of the MEDLINE strategy
Citation tracking
Science Citation Index
Scopus (1966 to present)
Additional search methods
Searching MEDLINE to identify a cross‐sectional sample of updated reviews indexed in November 2004
Reviewing reference lists
Contacting the Cochrane Collaboration's Reporting Bias Methods Group (RBMG)
The search strategy for the Cochrane Methodology Register will be:
updating and cumulative meta‐analysis in keywords
or
updat* or maintain* in All Fields and systematic review* or cumulative meta‐analys* or hta or clinical practice guideline or clinical guideline in All Fields
The search terms for MEDLINE will be:
1. Meta‐Analysis/
2. Practice Guidelines/
3. Technology Assessment, Biomedical/
4. exp "Review Literature"/
5. (systematic review$ or cumulative meta‐analys$ or hta or ((clinical or prevent$) adj2 guideline$)).mp.
6. or/1‐5
7. (updat$ or maintain$) adj5
8. Time Factors/
9. 6 and 8
10. An update.ti.
11. (update$ and maintain$).ab.
12. updat$.ti. and (updat$ or maintain$).ab.
13. updat$.ab. /freq=2
14. updating.ti.
15. or/11‐14
16. 15 not 10
17. 16 and (or/1‐4)
18. "value of information".mp.
19. or/7,9,17‐18
20. limit 19 to (editorial or letter)
21. cochrane database of systematic reviews.jn.
22. limit 19 to meta analysis
23. "systematic review of the literature".ti.
24. or/20‐23
25. 19 not 24
Data collection and analysis
Identifying studies
An experienced information specialist will conduct all searches. Two people will independently screen the titles and abstracts of the retrieved records for possible relevance. Full‐text reports for possibly eligible records will be obtained for more detailed assessment. At least two people will independently assess these reports for possible inclusion in the review. Any disagreements between the authors will be resolved by discussion.
Data extraction
Two people will use a pre‐specified data extraction form to independently extract data. Any disagreements will be resolved by discussion. The authors of included reports will be contacted for additional information as required. Data will be extracted on the type of study (e.g., descriptive, simulation, or comparative), publication year, objectives, the area of health care, methods for deciding when and how to update, underlying assumptions, rationale, comprehensiveness, complexity, and feasibility. Results of statistical tests applied in the context of updating systematic reviews will also be extracted.
Data analysis
The methods for updating systematic reviews will be compared descriptively with respect to their complexity, comprehensiveness, advantages and disadvantages. If the relevant information is available, we will compare time and resources spent for each method.
