Review Article
Inclusion of methodological filters in searches for diagnostic test accuracy studies misses relevant studies

https://doi.org/10.1016/j.jclinepi.2010.07.006Get rights and content

Abstract

Objective

To compare the performance of MEDLINE searches using index test(s) and target condition (subject searches) with the same searches combined with methodological filters for test accuracy studies.

Study Design and Setting

We derived a reference set of 506 test accuracy studies indexed on MEDLINE from seven systematic reviews that conducted extensive searches. We compared the performance of “subject” with “filtered” searches (same searches combined with each of 22 filters). Outcome measures were number of reference set records missed, sensitivity, number needed to read (NNR), and precision (Number of reference set studies identified for every 100 records screened).

Results

Subject searches missed 47 of the 506 reference studies; filtered searches missed an additional 21 to 241 studies. Sensitivity was 91% for subject searches and ranged from 43% to 87% for filtered searches. The NNR was 56 (precision 2%) for subject searches and ranged from 7 to 51 (precision 2–15%) for filtered searches.

Conclusions

Filtered searches miss additional studies compared with searches based on index test and target condition. None of the existing filters provided reductions in the NNR for acceptable sensitivity; currently available methodological filters should not be used to identify studies for inclusion in test accuracy reviews.

Section snippets

Background

What is new?

  • Using methodological search filters in addition to subject searches only produces a small reduction in the number needed to read compared with subject searches alone.

  • We have shown that inclusion of methodological filters misses additional studies compared with subject searches alone using a large reference set with a series of subject searches on a variety of topics.

  • Innovative graphics help to highlight the relative performance of multiple search filters.

  • Systematic reviewers should

Reference set

We identified seven systematic reviews of test accuracy studies, which were undertaken using similar and consistent literature search methods that facilitate comparison (Table 1) [24], [25], [26], [27], [28], [29], [30], [31]. Experienced information specialists carried out extensive well-documented literature searches for which full details and search results were available. All of the reviews employed “subject searches” that were not limited using methodological filters or search terms

Results

MEDLINE subject searches missed 47 (9%) of the 506 reference set records and missed from 0 to 10 (0–19%) records within reviews. Across reviews, the median sensitivity of subject searches was 91% (range, 81–100%). Searches that also included a methodological filter missed between 68 and 288 of the 506 reference set records, an additional 21–241 studies compared with the subject searches alone. For each filter, we calculated the median sensitivity of the filtered search across the seven reviews:

Discussion

Subject searches for test accuracy studies, based only on index test and target condition, miss records known to be indexed on MEDLINE despite being designed to maximize sensitivity. When subject searches are combined with methodological filters designed to identify test accuracy studies, even the best performing filters miss additional studies, with only small reductions in the NNR. Better performing filters were relatively brief: this is expected as combining search terms using OR tends to

References (34)

  • A.M. Fielding et al.

    Using Medline to achieve an evidence-based approach to diagnostic clinical biochemistry

    Ann Clin Biochem

    (2002)
  • N.L. Wilczynski et al.

    Indexing of diagnosis accuracy studies in MEDLINE and EMBASE

    AMIA Annu Symp Proc

    (2007)
  • R.B. Haynes et al.

    Developing optimal search strategies for detecting clinically sound studies in MEDLINE

    J Am Med Inform Assoc

    (1994)
  • L.M. Bachmann et al.

    Identifying diagnostic studies in MEDLINE: reducing the number needed to read

    J Am Med Inform Assoc

    (2002)
  • Critical Appraisal Skills Programme

    Search filter

    (2006)
  • W.L. Deville et al.

    Conducting systematic reviews of diagnostic studies: didactic guidelines

    BMC Med Res Methodol

    (2002)
  • M.C. Shipley

    Evidence-based filters for Ovid MEDLINE

    (2002)
  • Cited by (0)

    View full text