Evidence Based Medicine – How to interpret a Meta-Analysis

Dr Swapnil Pawar October 17, 2020 111

share close
  • cover play_arrow

    Evidence Based Medicine – How to interpret a Meta-Analysis
    Dr Swapnil Pawar

How to Interpret a Meta-Analysis

Blog Written by Dr Jose Chacko

  1. What is a meta-analysis and why do we need to perform a meta-analysis?

We live in the era of evidence-based medicine

Some important clinical questions remain unanswered despite several studies e.g., corticosteroids in sepsis; small studies may not be powered; blood sugar control differing conclusions

Put together studies systematically, attempt to come to a more definitive conclusion

A meta-analysis provides a single, overall measure of the treatment effect, enhancing the clinical interpretation of findings across several studies. You get results as “effect size”

  1. How meta-analysis is performed
  1. Ask the question PICO (population, intervention, control, outcomes): e.g., PPIs vs. H2 receptor blocker for stress ulcer prophylaxis
  2. Database search (Pubmed, Embase, Google Scholar, etc.) , using “search terms”. The search may include other sources (conference proceedings, hand search, contact leaders in the field)
  3. Time frame, published vs. unpublished; language restriction). Are you selecting RCTs alone or observational studies as well?
  4. Screening by title and abstract (two reviewers)
  5. Choose papers, read full text (two reviewers, a third reviewer to arbitrate if there is a difference of opinion)
  6. Exclude papers (duplicates, not relevant, studies excluded)
  7. Assess the quality of papers
  8. Assess heterogeneity
  9. Estimate the summary effect size in the form of Odds Ratio and using both fixed and random effects models and construct a forest plot

Publication bias using a funnel plot

Subgroup analysis (steroids: age, severity, dose, type)

Register the study protocol (PROSPERO)

Effect size

Studies must have common outcomes that can be compared. Continuous data (e.g., ICU days): standardized mean difference; dichotomous data (e.g., mortality): odds ratio, hazard ratio, risk ratio (relative risk) (with confidence interval: wide (less precise) vs. narrow(more precise)


When the magnitude and direction of the effect sizes among the studies are similar, heterogeneity is less likely and meta-analysis may be appropriate. Conversely, when study results vary, a high degree of heterogeneity is possible and a meta-analysis may not be appropriate.

  1. How one should evaluate the bias involved in RCTs

Bias is a deviation from the true result

Bias domain


Bias arising from the randomization process

(selection bias)

  • Baseline differences between intervention groups suggest a problem with the randomization process.
  • Was the allocation sequence was adequately concealed?

Bias due to deviations from intended interventions

(performance bias)


  • Blinding: carers and people delivering the interventions were aware of participants’ assigned intervention during the trial. If the outcome is objective (mortality), absence of blinding may have less of an effect
  • Were important non-protocol interventions were balanced across intervention groups?
  • Did study participants adhered to the assigned intervention regimen?

Bias due to missing outcome data

(attrition bias)

  • Loss to follow up
  • Missing data 

Bias in measurement of the outcome

(detection bias)


  • Was the method of measuring the outcome appropriate?
  • Were outcome assessors aware of the intervention?

Bias in selection of the reported result

(reporting bias)


  • Non-significant time frame – e.g. 1-week mortality
  • Results confined to narrow subgroups of patients
  • Were only positive outcomes reported?

Cochrane ROB tool (low, high, or unclear risk of bias) is used to assess RCTs

  1. What is Forrest plot and what are the key terminologies associated with it. 

A forest plot of a meta-analysis typically includes

  1. The numerical value of the treatment effect and variability for each individual study (e.g., odds ratio with confidence interval). The effect size (e.g., odds ratio) is the square in the middle with the horizontal line representing the CI
  2. The size of the square represents “weight” of the study (how much did the study contribute to the pooled estimate)?
  3. The modelling technique assumed (random or fixed). Random effects model if there is a high degree of heterogeneity, fixed effect if the degree of heterogeneity is low. It mentions the degree of heterogeneity, using I2 
  4. The “line of no effect,” which is the vertical line (the stem of the tree)
  5. The numerical estimate of overall treatment effect at the bottom which appears as a diamond. The forest plot offers a quick visual assessment of the individual studies, assessment of heterogeneity, and the overall treatment effect, represented by the diamond at the bottom

Fixed and random effects model: A fixed-effect model assumes that the treatment effect between studies is similar and differences between studies occur by chance alone. In other words, the treatment effect is assumed to be common or “fixed”. On the other hand, the random-effects model assumes that the treatment effect is variable between studies.

Heterogeneity is evident by visualizing forest plot; you will find that the odds ratio or the SMD is dispersed widely across the line of effect (the vertical line). The I2 (Higgins) is used to assess heterogeneity. The I2 value varies between 0-100, with higher values indicating increasing heterogeneity. There are not strict cut-offs; however, 25% or lower suggests low heterogeneity, 25-50% suggests medium heterogeneity, and 75% or more suggests high heterogeneity. A fixed-effects model is used for low heterogeneity and a random-effects model is used for a high degree of heterogeneity.

Rate it
Previous episode
eCritCare Podcast
share playlist_add
  • 601


Evidence-Based Medicine Part 1- How to Interpret an RCT

Dr Swapnil Pawar October 4, 2020

Interpreting an RCT Blog written by – Dr Jose Chacko & Dr Swapnil Pawar 3 Key areas – the validity of the trial methodology; the magnitude and precision of the […]

Read more trending_flat