Risk of bias assessment (sometimes called "quality assessment," "validity assessment," or "critical appraisal") helps to establish transparency of evidence synthesis/systematic review results and findings. A risk of bias assessment is often performed for each included study in your review.
Evidence syntheses/systematic reviews strive to eliminate bias in their findings. Individual studies that are included in a synthesis may include biases in their results or conclusions, for example design flaws that raise questions about validity of findings or an overestimate of intervention effect.
Risk of bias assessment generally is not required with evidence synthesis methods outside of systematic reviews. For example, scoping reviews typically don't include a risk of bias assessment. Because a scoping review is not aiming to make a clinical recommendation based on its findings, the included sources are typically not critically appraised.
Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018 Oct 2;169(7):467–73
There are several tools based on study type to assess risk of bias in your included studies. Remember, each study must be assessed individually - if you have 20 included studies, that's 20 risk of bias assessments. The tool you choose will depend on the types of studies you include in your review.
The LATITUDES Library is a searchable library of validity assessment tools designed for use in evidence syntheses. The website also provides access to training on the process of validity assessment. Below are some examples of assessment tools:
| Randomized trials |
|
RoB2 - Revised Cochrane Risk-of-Bias tool for randomized trials
|
| Non-randomized studies |
| ROBINS-I - Cochrane's Risk of Bias In Non-randomized Studies – of Interventions |
| ROBINS-E - Cochrane's Risk of Bias In Non-randomized Studies - of Exposure |
| The Newcastle-Ottawa Scale (NOS) for assessing the quality of non-randomized studies |
| NIH Study Quality Assessment Tools - for Controlled Intervention, Observational Cohort and Cross-Sectional, Case-Control, Case Series Studies |
| QUADAS – Quality Assessment of Diagnostic Accuracy Studies |
| QUIPS – Quality in Prognosis Studies |
| The Joanna Briggs Institute Prevalence Critical Appraisal Tool - for studies reporting prevalence data |
| PROBAST - Prediction model Risk Of Bias ASsessment Tool |
| Randomized and Non-randomized studies |
| Downs and Black Checklist for Quality Assessment |
Risk of bias assessments are usually presented in the manuscript as a table based on the findings from the risk of bias tool used. For each study, the individual domains assessed and overall rating are marked as "Low risk of bias" (green +); "Some concerns" (yellow -); or "High risk of bias" (red x).
robvis – creates publication quality risk-of-bias assessment figures.
Upload data from your risk of bias assessment into the app using robvis templates. Currently supports the following risk of bias assessment tools: RoB2, ROBINS-I, ROBINS-E, QUADAS, QIPS, and a generic form.