Skip to Main Content

Systematic Reviews: Systematic Review Defined

Collaborative support for researchers undertaking systematic reviews

What is a Systematic Review?

A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question.  It  uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made.

The key characteristics of a systematic review are:

  • a clearly stated set of objectives with pre-defined eligibility criteria for studies
  • an explicit, reproducible methodology
  • a systematic search that attempts to identify all studies that would meet the eligibility criteria
  • an assessment of the validity of the findings of the included studies
  • a systematic presentation, and synthesis, of the characteristics and findings of the included studies

Meta-analysis is the use of statistical methods to summarize the results of independent studies. Many systematic reviews contain meta-analyses. By combining information from all relevant studies, meta-analyses can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review. They also facilitate investigations of the consistency of evidence across studies, and the exploration of differences across studies.

Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. Updated March 2011. Section 1.2.2. 

Systematic Reviews, Meta-analyses and Topic Reviews Compared

  Systematic review Meta-Analysis Topic/Literature review

Review question

Starts with clear question to be answered or hypothesis to be tested

Starts with clear question to be answered or hypothesis to be tested

May also start with clear question to be answered, but more often involve general discussion of subject with no stated hypothesis

Background

Provide summaries of the available literature on a topic

 

 

Searching for relevant studies

Strive to locate all relevant published and unpublished studies to limit impact of publication and other biases

Strive to locate all relevant published and unpublished studies to limit impact of publication and other biases

May use funnel plot to assess completeness

Do not usually attempt to locate all relevant literature. Strategy not explicitly stated

Deciding which studies to include and exclude

Involve explicit description of what types of studies are to be included to limit selection bias

Involve explicit description of what types of studies are to be included to limit selection bias on behalf of reviewer

Usually do not describe why certain studies are included and others excluded

Assessing study quality

Examine in systematic manner methods used in primary studies, and investigate potential biases in those studies and sources of heterogeneity between study results

Quantitative analysis of measures of effect, accounting for heterogeneity

Often do not consider differences in study methods or study quality

Results and data synthesis

Base their conclusions on those studies which are most methodologically sound

Base their conclusions on those studies which are most methodologically sound

Often do not differentiate between methodologically sound and unsound studies. May also be influenced by the reviewer's theories, needs and beliefs

Discussion Written by an expert or group of experts with a detailed and well-grounded knowledge of the issues Written by an expert or group of experts with a detailed and well-grounded knowledge of the issues

 

Adapted from:

Petticrew M. Systematic reviews from astronomy to zoology: myths and misconceptions. BMJ. 2001 Jan 13;322(7278):98-101.

Bettany-Saltikov, J. (2010). Learning how to undertake a systematic review: Part 1. Nursing Standard, 24(40): 47-55.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009 Jun;26(2):91-108.

Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. Updated March 2011. Section 1.2.2.

Strengths and Weaknesses of Systematic Reviews

Systematic Review

Meta-analyses Narative Review

Strengths

Reduces reviewer bias by use of objective, reproducible criteria to select relevant individual publications and assess their validity

Saves decision-makers time

Aggregates  effects from several studies

Small or inconclusive studies lacking in statistical significance can make a contribution to the larger picture

Saves decision-makers time

Generally comprehensive and cover a wide range of issues within a given topic

Review articles can be used to bring one up-to-date on a particular topic

Weakness

Difficult and time consuming particularly when identifying appropriate studies

Narrow focus and prescribed methods do not allow for comprehensive coverage

Difficult and time consuming particularly when identifying appropriate studies

Cannot eliminate (without registration of all trials) publication bias wherein positive findings get published and negative ones do not

Subjective (terms of what studies to include)

Selection bias (in terms of what studies to include)

Adapted from:

Petticrew M. Systematic reviews from astronomy to zoology: myths and misconceptions. BMJ. 2001 Jan 13;322(7278):98-101.

Bettany-Saltikov, J. (2010). Learning how to undertake a systematic review: Part 1. Nursing Standard, 24(40): 47-55.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009 Jun;26(2):91-108.

Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. Updated March 2011. Section 1.2.2.

Relevant Guides