Skip Navigation Bar
NLM Technical Bulletin Header
Article Navigation Bar Table of Contents NLM Technical Bulletin Home Page Back Issues
 February 15, 2002 [posted]
 New PubMed® Filter: Systematic Reviews

drop cap graphic of the letter A common complaint among those looking for biomedical information is that there is too much of it — too many journals, too many articles, and too little time. Possibly as a result of this, systematic review articles are becoming more popular, especially with clinicians, because they summarize and evaluate information that can help in decision-making. Cook DJ, et al. highlight the importance of these articles, "Used increasingly to inform medical decision making, plan future research agendas, and establish clinical policy, systematic reviews may strengthen the link between best research evidence and optimal health care"[1].

In November 2001, a Systematic Reviews search filter was added to PubMed on the Clinical Queries screen to make it easier to retrieve citations for these distinctive articles.

Review vs. systematic review. What is the difference?
Whereas review articles are generally an examination of the literature from the author's perspective, a systematic review can be thought of as a study design that includes some or all of the following characteristics:
  1. One or more questions or hypotheses are posed at the start of the review.
  2. Publications/study results in the subject area are collected, sometimes limited to a particular type (e.g., literature reviews, randomized controlled trials), and are often obtained from specific sources (e.g., databases, bibliographies).
  3. Criteria are used to include or exclude studies found in #2 for the actual review. This step is often referred to as the "study selection."
  4. Data extracted from the selected studies are combined and compared, or if the data cannot be combined, the strength of the evidence is assessed and used to evaluate results.
  5. Conclusions are made based on results and/or the presence or absence of supporting evidence.
Additional Topics Included
Meta-analyses are often considered a type of systematic review and are distinguished by the application of quantitative and statistical methods for combining and examining results. Often, a systematic review will include a meta-analysis.

In addition to meta-analyses, the Systematic Reviews filter also retrieves citations to other articles of interest to health professionals such as reviews of clinical trials, evidence-based medicine, consensus development conferences, guidelines, and citations to articles from journals specializing in clinical review studies.

The abstracts for the articles cited below provide good examples of systematic reviews:

Bronfort G, Assendelft WJ, Evans R, Haas M, Bouter L.
Efficacy of spinal manipulation for chronic headache: a systematic review.
J Manipulative Physiol Ther. 2001 Sep;24(7):457-66. [See Abstract.]

Phillips KA, Veenstra DL, Oren E, Lee JK, Sadee W.
Potential role of pharmacogenomics in reducing adverse drug reactions: a systematic review.
JAMA. 2001 Nov 14;286(18):2270-9. [See Abstract.]

Search Systematic Reviews from the Clinical Queries screen
PubMed's Systematic Reviews filter is available from the Clinical Queries screen (link from sidebar). This screen has been modified so that users can apply either a Clinical Queries filter or the Systematic Reviews filter to their search terms (see Figure 1.)
  Systematic Reviews filter on Clinical Queries screen

To use this new filter, select Systematic Reviews by clicking on the radio button, enter a topic, and click on the Go button.

After running a Systematic Reviews search, PubMed's Limits, e.g., English language, can be easily applied to the retrieval, if desired.

Alternatively, this filter can be used directly in a PubMed search as "systematic [sb]."
Example: diabetic retinopathy AND systematic [sb]

The Strategy Used for the Filter
The filter works by combining the user's search term(s) with citations that were retrieved by PubMed's Systematic Reviews search strategy.

The strategy has 3 parts:

  1. The Basic Strategy:
    There is no Publication Type in MeSH for "systematic reviews." (These articles are generally indexed with the Publication Type Review, and/or Meta-Analysis.) Therefore, the basic strategy searches for variations of the term, "systematic review" in the title and abstract. It also includes Publication Types and free-text searching for Meta-Analysis, Guidelines, and Consensus Development Conference. The term, "evidence-based medicine" is included although this filter does not attempt to provide comprehensive retrieval in that area. Review [PT] is not used as this would retrieve "non-systematic" reviews.

    The basic strategy also includes some journal titles to retrieve citations in journals known to publish systematic reviews, e.g., Cochrane database of systematic reviews.

  2. The Supplemental Strategy
    There are many articles in which the study methodology describes a systematic review or meta-analysis but the author has not used these terms in the title or abstract, nor are they published in a specialized journal. In an attempt to include references to some of these studies, the strategy was supplemented with additional search terms that rely on free-text searching for a combination of terms. This type of search can often result in undesirable retrieval, so the supplemental strategy is fairly conservative. Some search terms or phrases that may seem appropriate have not been included because they would retrieve an unacceptable number of citations that are not systematic reviews.

    The supplemental part of the strategy is made up of three sets of search terms:
    • Terms that imply something has been done "systematically."
    • Terms used for the concepts of a review or overview.
    • Terms for sources of data, e.g., literature, databases, trials.

    The retrieval from the three sets are combined using the AND operator.

  3. Bringing Basic and Supplemental Together:
    • The basic strategy is ORed with the supplemental strategy.
    • Letters to the editor, editorials, case reports, etc., are excluded as these are generally not of interest to clinicians looking for systematic reviews.

Indexing status affects retrieval. For example, a number of publication types are excluded by the strategy, e.g., Letter. This means that a citation may be retrieved while it is in process but excluded once it is indexed. Alternatively, the filter will not retrieve some relevant citations until they are indexed.

Other sources of information explaining methods of locating systematic review articles have been provided.

We hope this filter will prove to be helpful to clinicians, health services researchers, and others interested in articles examining and evaluating clinical modalities.

Work Cited:

 1. Cook DJ, Mulrow CD, Haynes, RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med 1997 Mar 1;126(5):376-80. [PMID: 9054282]


Mulrow C, Cook D, editors. Systematic reviews: synthesis of best evidence for health care decisions. Philadelphia: American College of Physicians, 1998.

Undertaking systematic reviews of research on effectiveness: CRD's guidance for those carrying out or commissioning reviews. CRD Report Number 4 (2nd Edition) March 2001 [Internet]. NHS Centre for Reviews and Dissemination, University of York, York, United Kingdom. Available from: Search Undertaking systematic reviews of research on effectiveness: CRD's guidance for those carrying out or commissioning reviews, Year 2001.

By Annette M. Nahin
MEDLARS Management Section

Black separating line

Nahin AM. New PubMed® Filter: Systematic Reviews. NLM Tech Bull. 2002 Jan-Feb;(324):e7.


Article Navigation Bar NLM Technical Bulletin Home Page Back Issues Index Previous Page Next Article
U.S. National Library of Medicine, 8600 Rockville Pike, Bethesda, MD 20894
National Institutes of Health, Department of Health & Human Services
Copyright, Privacy, Accessibility, Viewers and Players
Freedom of Information Act (FOIA)
HHS Vulnerability Disclosure