Skip to main content

Systematic Reviews: Reviewing Studies

High-quality systematic reviews are designed to identify all relevant published and unpublished evidence and provide an unbiased synthesis of the findings.

Reviewing Studies

At least two team members must independently evaluate all the studies. There are two rounds of review.  In the first review, members vote on whether to include/exclude each study based solely on the titles and abstracts. The reputations of the authors and journals should not influence their decision.   You don't need to record the reasons why you decided to exclude studies. After both team members have evaluated the studies, votes are compared and differences are settled in a pre-specified way. Often, a third member is called on to decide in order to prevent bias.

In the second round of reviews, once again, at least two team members must independently evaluate the remaining studies but this time, they appraise the full-text. Reviewers must record the reasons for excluding individual studies based on the inclusion/exclusion criteria they chose when formulating their protocol.  Once again, if there are any differences of opinion, a third member is often called on to decide in order to prevent bias.  Usually only a very small percentage of the initial documents retrieved are included in the final systematic review. Common estimates range from 1-3%.

  • Excel is the most basic tool for screening citations. You can export from EndNote and create customized workbooks and spreadsheets.
  • Rayyan - a free online tool that screens, analyzes, and extracts data. You can can export citations from EndNote or directly from the databases themselves. It is easily shareable among several reviewers and allows independent title/abstract screening and coding. It uses tagging and filtering to code and organize references.s  creen, analyze, and extract data from systematic reviews.Rayyan also has a free mobile app (iOS/Android).
  • Covidence - easily sharable among several reviewers and allows for independent title/abstract screening, full text screening, data extraction, and risk of bias assessment.  It is now required by reviewers doing Cochrane systematic reviews.  It currently allows a free trial to do a single review shared between two reviewers.
  • DistillerSR - allows for the management of the entire systematic review process with multiple independent reviewers.  Requires a monthly subscription fee.
  • Abstrackr - created by Brown University, Abstrackr aids in screening abstracts.
  • RevMan: - a free online tool used by Cochrane reviewers to manage the data extraction and analysis process.