- Library Home
- Research Guides
- Evidence-Based Medicine
- Appraise the Evidence

This is a libguide for praticing evidence-based medicine.

- Users' Guides to the Medical Literature: a Manual for Evidence-Based Clinical Practice, 3E byISBN: 9780071790710Publication Date: 2014
- Finding What Works in Health Care: Standards for Systematic Reviews byISBN: 9780309164252Publication Date: 2011
- Evidence-Based Practice Toward Optimizing Clinical Outcomes byISBN: 9783642050251Publication Date: 2010-06-17

The critical appraisal process hinges on three questions that apply to any study:

**1. Are the results of the study valid?** (**Validity**)

**2. What are the results? (Reliability)**

**3. Will the results help me in caring for my patients? (Applicability)**

The critical appraisal process provides clinicians with the tools to interpret the quality of studies and determine the applicability of the synthesis of multiple studies' results to their patients.

The **validity**** **of a study refers to whether the results of the study were obtained via sound scientific methods. Bias (defined as the systematic deviation from the truth) and/or confounding variables may compromise the validity of the finding. The** reliability **of the study's results are determined by the size of the intervention's effect (the effect size) and how precisely that effect was estimated. This part of critical appraisal examines the numerical data reported in the results section of a study. When critically appraising the the usefulness of a study for clinical decision making, a basic aspect of **applicability** is to evaluate the study's patients in comparison with the patients to whom the evidence would be applied.

- CASP ChecklistsThis set of eight critical appraisal tools are designed to be used when reading research, these include tools for Systematic Reviews, Randomised Controlled Trials, Cohort Studies, Case Control Studies, Economic Evaluations, Diagnostic Studies, Qualitative studies and Clinical Prediction Rule.
- Appraisal of Guidelines for Research and Evaluation (AGREE))The Appraisal of Guidelines for Research and Evaluation (AGREE) Instrument evaluates the process of practice guideline development and the quality of reporting.
- Critical Appraisal Worksheet by CEBMCentre for Evidence-based Medicine provides useful tools and downloads for the critical appraisal of different types of medical evidence. Example appraisal sheets are provided together with several helpful examples.
- Knowledge Translation Program (Toronto)Critical appraisal worksheets from centre for Evidence-Based Medicine Toronto, including the worksheets for diagnosis, harm, prognosis, systematic review and therapy.
- Equator NetworkThe EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network is an international initiative that seeks to improve the reliability and value of published health research literature by promoting transparent and accurate reporting and wider use of robust reporting guidelines.

- RobotReviewer: A system for automatically assessing bias in clinical trials. Free to use.

**Absolute Risk (AR)** = incidence

The observed or calculated probability of an event in the population under study.

**Experimental Event Rate (EER)** = a/a+b [i.e. Risk in exposed]

**Control Event Rate (CER)** = c/c+d [i.e. Risk in unexposed]

**Relative Risk (RR)** = **EER/CER **= (a/a+b)/(c/c+d) =

The ratio of the probability of developing, in a specified period of time, an outcome among those receiving the treatment of interest or exposure to a risk factor, compared with the probability of developing the outcome if the intervention or risk factor is not present.

**Relative Risk Reduction (RRR)** = **CER-EER/CER** or = **1-RR**

The extent to which a treatment reduces a risk, in comparison with patients not receiving the treatment of interest.

**Absolute Risk Reduction (ARR)** = **CER-EER **[also referred to as the** risk difference (RD)**]

The difference in the absolute risk (rates of adverse events) between study and control populations.

**Number Needed to Treat (NNT) = 1/ARR** Video by Terry Shaneyfelt

The number of patients who must be exposed to an intervention before the clinical outcome of interest occurred; for example, the number of patients who must be treated to prevent one adverse outcome. **NNT** is a value that can permit all stakeholders in the clinical decision to better understand the likelihood of developing the outcome if a patient has a given intervention or condition.

**Odds**

A proportion in which the numerator contains the number of times an event occurs and the denominator includes the number of times the event does not occur.

**Odd Ratio (OR) = **(a/b)/(c/d)=ad/bc

A measure of the degree of association; for example, the odds of exposure among the cases compared with the odds of exposure among the controls. Note: **OR **and **RR** can be very similar when outcomes or events are rare. As the outcomes or event rate increase, the value will diverge.

**Confidence Interval (CI)** Video by Terry Shaneyfelt

The range in which the true effects lies with a given degree of certainty. In other words, the **CI** provides clinicians a range of values in which they can be reasonably confident (e.g., 95%) that they will find a result when implementing the study findings. In general, narrower** CIs** are more favorable than wider **CIs**; where confidence intervals are wide, they indicate less precise estimates of effect. When the confidence interval crosses the point of no effect (e.g., for **OR** or **RR**, no effect=1; for effect size, no effect=0), it demonstrates no statistical significance.

*p *Value

The probability that any particular outcome would have occurred by chance. A ** p value** of 0.05 or less would be considered a statistically significant result in healthcare research. Considered to be inferior to

**Sensitivity** = a/(a + c)

Sensitivity** **measures the proportion of patients with the disease who also test positive for the disease in this study. It is the probability that a person with the disease will have a positive test result.

**Specificity** = d/(b + d)

Specificity measures the proportion of patients without the disease who also test negative for the disease in this study. It is the probability that a person without the disease will have a negative test result.

**Positive Predictive Value (PPV)** = a / a+b

**PPV** is the probability that subjects with a positive screening test truly have the disease. **PPV **can also be calculated as **PPV** = sensitivity x prevalence / sensitivity x prevalence + (1-sensitivity) x (1-prevalence)

**Negative Predictive Value (NPV)** = d / c +d** **

**NPV **is the probability that subjects with a negative screening test truly don't have the disease. **NPV **can also be calculated as **NPV** = specificity x (1-prevalence) / (1-sensitivity) x prevalence + specificity x (1-prevalence)

Note that the PPV and NPV is not intrinsic to the test - it depends also on the **prevalence**. NPV and PPV should only be used if the prevalence of patients being evaluated is equivalent to the prevalence of the diseases in the reported population.

**Likelihood Ratios (LR): **The LR is the probability of a given test result in a patient with the target disorder divided by the probability of that same result in a person without the target disorder. Unlike the sensitivity and specificity, LRs are immune to prevalence.

**Positive Likelihood Ratio (LR+)** = a/(a+c) / b/(b+d)

The probability that an individual with the target disorder has a positive test probability divided by the probability that an individual without the target disorder has a positive test. In other words, **LR+ **= true positivity rate / false positivity rate, which is the same as sensitivity / (1- specificity).

**Negative Likelihood Ratio (LR-)** = c/(a+c) / d(b+d)

The probability of an individual with the target disorder having a negative test divided by the probability of an individual without the target disorder having a negative test. In terms of sensitivity and specificity, **LR-** = (1-sensitivity) / specificity

**Forest Plots**

How to interpret a forest plot. Video by Terry Shaneyfelt