Skip to Main Content

Systematic Reviews: Appraising Studies

Appraising Studies

You need to evaluate the quality of each study you include in your systematic review. This critical appraisal is often called "quality assessment" or "risk of bias assessment."  Your evaluations are shown in a table format in your article. There are several tools to help you evaluate various types of studies and create a table.  

Evaluation Tools for Controlled Trials

Cochrane's ROB 2 Risk of Bias Tool evaluates randomized controlled trials on seven items: 

  • Random sequence generation
  • Allocation concealment
  • Blinding of participants and personnel
  • Blinding of outcome assessment
  • Incomplete outcome data
  • Selective reporting
  • Other possible sources of bias

For more information on risk of bias, see the Cochrane Handbook, Chapter 8: Assessing risk of bias in included studies. 

Jadad Scaleassigns trials a score of 0-5 based on the quality of randomization, blinding, and withdrawls

Robvis: a free online tool to create high-quality tables that summarize the risk of bias across studies with red, green, and yellow circles. 

Evaluation Tools for NON Randomized Controlled Trials

There are dozens of Risk of Bias Tools for various types of studies. For a comprehensive list, download the Excel spreadsheet at the Risk of Bias Tool Repository.  A few of the more common ones are:

CASP Checklists  appraise RCTs and non-RCTs such as cohort, case control, economic evaluations, diagnostic studies, qualitative studies, and clinical prediction rules. 

Cochrane's ROBINS-I tool evaluates cohort studies, quasi-randomized trials, case-control studies, cross-sectional studies, interrupted time series, and controlled before-after studies. 

Newcastle-Ottawa Scale:   appraises case-control or cohort studies. 

NIH Study Quality Assessment Tools evaluate controlled intervention studies, systematic reviews, meta-analyses, observational cohort and cross-sectional studies, case-control studies, and before-after (pre-post) studies with no control groups.

JBI Critical Appraisal Tools: evaluate cross-sectional, case control, case reports, case series, cohort, diagnostic test accuracy, economic evaluations, economic, prevalence, qualitative, quasi-experimental, and systematic reviews. 

QuADS Quality Appraisal for Diverse Studies: an appraisal tool for systematic reviews which include a variety of mixed and multi-method studies

Guidelines for Assessing Studies

Assessing the Risk of Bias in Systematic Reviews of Health Care Interventions (from AHRQ Methods Guide for Comparative Effectiveness Reviews)