A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made.
The key characteristics of a systematic review are:
Meta-analysis is the use of statistical methods to summarize the results of independent studies. Many systematic reviews contain meta-analyses. By combining information from all relevant studies, meta-analyses can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review. They also facilitate investigations of the consistency of evidence across studies, and the exploration of differences across studies.
A focused, well-defined clinical question is essential before beginning any steps in the systematic review process. Starting with a well-articulated idea will help dictate the direction you will take in all the steps that follow.
Use the PICO format to formulate your question.
It’s important to find out if there is already a recent review or protocol on your topic. A fresh point of view helps your work to be accepted by a scholarly journal. Will your article be redundant or is an update warranted?
Systematic review standards, best practices, and guidelines, such as PRISMA, The Cochrane Handbook, and The National Academies of Sciences (previously Institute of Medicine) all require or recommend registering a protocol before performing a systematic review. The main purpose of registering a protocol is twofold: it aims to reduce unplanned duplication of systematic reviews and also provides transparency in the review process, therefore minimizing reporting bias. These protocols are registered with PROSPERO, an international prospective register of systematic reviews.
This is also where you will define your inclusion/exclusion criteria, a crucial step in the SR process that sets the boundaries for the systematic review. Inclusion criteria are characteristics that the articles must have if they are to be included in the study, while exclusion criteria are those characteristics that disqualify articles from inclusion in the study. These can include, but are not limited to: geographic location, participants, reported outcomes, exposure of interest, setting, study design, type of publication.
A thorough and comprehensive search is the cornerstone to a systematic review. Systematic reviews aim to cast a wide net in order to find all available literature on a given topic. Therefore, systematic review searches can be complex and lengthy, often yielding thousands of results.
This means you’ll need to use a combination of controlled vocabulary (e.g., MeSH for MEDLINE, EMTREE for Embase) and a comprehensive list of keywords.
Best practice states that a systematic review must search at least 3 different databases. Different databases cover different literature, so in order to be comprehensive several should be included in your systematic review. Three multidisciplinary databases that should at least be searched are:
Depending on your topic, you may want to search other discipline specific databases as well. Some that we subscribe to at WCM are:
REMEMBER: Not all databases are searched in the same way. Different platforms have different search functionalities and controlled vocabulary. Once you’ve chosen your databases, be sure to look at their advanced search functions to see how to build your search in each database.
There is some overlap between any of the databases you will choose. These duplicates need to be removed before you begin your screening process. Once you’ve exported the results from each database, this can be done in a tool like EndNote, or in a systematic review screening tool like Covidence. After meeting with a faculty librarian for an initial consultation, you will be provided access to Covidence.
At this first round of screening, you will be looking at just the titles and the abstracts against your inclusion/exclusion criteria to determine eligibility. In order to reduce selection bias, at least 2 reviewers must independently assess each record, with discrepancies resolved by consensus or a 3rd reviewer. This is where the bulk of the records will be screened out.
For those records that you’ve selected to go to the next stage of screening, you’ll need to track down the PDFs and attach them to the records in Covidence. If you have several hundred, there is a way to do this in bulk, but it requires EndNote desktop client.
At this second round of screening you will be looking at the full article to determine eligibility; this is where you will decide which articles will be included in your review. In addition to inclusion/exclusion agreement, reviewers must also agree on the reason why a record was excluded. This information is reported in the PRISMA flow diagram.
Once you’ve determined which articles will be included in your review, you’ll then pull the bibliographies and citing articles of those studies and perform another, final round of screening. Search for the article title in a database like Scopus, and you'll find reference cited lists (backward) as well as who has cited the paper (forward).
According to your pre-defined data points, you’ll now begin pulling data from your final list of included articles. This should also be performed by 2 reviewers independently in order to reduce bias. This can be done with Excel or Covidence.
Risk of bias assessment (sometimes called "quality assessment" or "critical appraisal") helps to establish transparency of evidence synthesis results and findings. A risk of bias assessment should be performed for each included study in your review. There are various tools to help with this process, often sorted by study type.
As systematic reviews often take a year or more to complete, it’s necessary to make sure that your searches include the most current literature before you start writing your manuscript. If it has been over a year, searches should be re-run and any new records screened. Many journals require that SR searches be run within a year or even 6 months of submission.
Meta-analysis is the statistical procedure for combining data from multiple studies. When the treatment effect (or effect size) is consistent from one study to the next, meta-analysis can be used to identify this common effect. When the effect varies from one study to the next, meta-analysis may be used to identify the reason for the variation. Not all systematic reviews can be meta-analyses; it depends on the data available in the studies.
You’re now ready to begin writing your review. There are several guidelines and reporting checklists to help make sure your review is well reported and methodologically sound.