Evidence-Based Medicine: Appraise the Evidence

The critical appraisal process hinges on three questions that apply to any study:

1. Are the results of the study valid? (Validity)

2. What are the results? (Reliability)

3. Will the results help me in caring for my patients? (Applicability)

The critical appraisal process provides clinicians with the tools to interpret the quality of studies and determine the applicability of the synthesis of multiple studies' results to their patients.

The validity of a study refers to whether the results of the study were obtained via sound scientific methods. Bias (defined as the systematic deviation from the truth) and/or confounding variables may compromise the validity of the finding. The reliability of the study's results are determined by the size of the intervention's effect (the effect size) and how precisely that effect was estimated. This part of critical appraisal examines the numerical data reported in the results section of a study.  When critically appraising the the usefulness of a study for clinical decision making, a basic aspect of applicability is to evaluate the study's patients in comparison with the patients to whom the evidence would be applied.

• RobotReviewer: A system for automatically assessing bias in clinical trials. Free to use. 

Absolute Risk (AR) = incidence

The observed or calculated probability of an event in the population under study.

Experimental Event Rate (EER) = a/a+b  [i.e. Risk in exposed]

Control Event Rate (CER) = c/c+d [i.e. Risk in unexposed]

Relative Risk (RR) = EER/CER = (a/a+b)/(c/c+d) =

The ratio of the probability of developing, in a specified period of time, an outcome among those receiving the treatment of interest or exposure to a risk factor, compared with the probability of developing the outcome if the intervention or risk factor is not present.

Relative Risk Reduction (RRR) = CER-EER/CER or = 1-RR

The extent to which a treatment reduces a risk, in comparison with patients not receiving the treatment of interest.

Absolute Risk Reduction (ARR) = CER-EER [also referred to as the risk difference (RD)]

The difference in the absolute risk (rates of adverse events) between study and control populations.

Number Needed to Treat (NNT) = 1/ARR  Video by Terry Shaneyfelt

The number of patients who must be exposed to an intervention before the clinical outcome of interest occurred; for example, the number of patients who must be treated to prevent one adverse outcome. NNT is a value that can permit all stakeholders in the clinical decision to better understand the likelihood of developing the outcome if a patient has a given intervention or condition.

Odds

A proportion in which the numerator contains the number of times an event occurs and the denominator includes the number of times the event does not occur.

Odd Ratio (OR) = (a/b)/(c/d)=ad/bc

A measure of the degree of association; for example, the odds of exposure among the cases compared with the odds of exposure among the controls. Note: OR and RR can be very similar when outcomes or events are rare. As the outcomes or event rate increase, the value will diverge.

Confidence Interval (CI)   Video by Terry Shaneyfelt

The range in which the true effects lies with a given degree of certainty. In other words, the CI provides clinicians a range of values in which they can be reasonably confident (e.g., 95%) that they will find a result when implementing the study findings. In general, narrower CIs are more favorable than wider CIs; where confidence intervals are wide, they indicate less precise estimates of effect. When the confidence interval crosses the point of no effect (e.g., for OR or RR, no effect=1; for effect size, no effect=0), it demonstrates no statistical significance.

p Value

The probability that any particular outcome would have occurred by chance. A p value of 0.05 or less would be considered a statistically significant result in healthcare research. Considered to be inferior to Confidence Intervals in determining significance of studies.

Sensitivity = a/(a + c)

Sensitivity measures the proportion of patients with the disease who also test positive for the disease in this study. It is the probability that a person with the disease will have a positive test result.

Specificity = d/(b + d)

Specificity measures the proportion of patients without the disease who also test negative for the disease in this study. It is the probability that a person without the disease will have a negative test result.

Positive Predictive Value (PPV) =  a / a+b

PPV is the probability that subjects with a positive screening test truly have the disease. PPV can also be calculated as PPV = sensitivity x prevalence / sensitivity x prevalence + (1-sensitivity) x (1-prevalence)

Negative Predictive Value (NPV) =  d / c +d

NPV is the probability that subjects with a negative screening test truly don't have the disease. NPV can also be calculated as NPV = specificity x (1-prevalence) / (1-sensitivity) x prevalence + specificity x (1-prevalence)

Note that the PPV and NPV is not intrinsic to the test - it depends also on the prevalence. NPV and PPV should only be used if the prevalence of patients being evaluated is equivalent to the prevalence of the diseases in the reported population.

Likelihood Ratios (LR): The LR is the probability of a given test result in a patient with the target disorder divided by the probability of that same result in a person without the target disorder. Unlike the sensitivity and specificity, LRs are immune to prevalence.

Positive Likelihood Ratio (LR+) = a/(a+c) / b/(b+d)

The probability that an individual with the target disorder has a positive test probability divided by the probability that an individual without the target disorder has a positive test. In other words, LR+ = true positivity rate / false positivity rate, which is the same as sensitivity / (1- specificity).

Negative Likelihood Ratio (LR-) = c/(a+c) / d(b+d)

The probability of an individual with the target disorder having a negative test divided by the probability of an individual without the target disorder having a negative test. In terms of sensitivity and specificity, LR- = (1-sensitivity) / specificity

Forest Plots

How to interpret a forest plot. Video by Terry Shaneyfelt