Efficacy vs. Effectiveness

Understanding the difference between trial results and real-world performance

These two terms are often confused but measure different things. Understanding the distinction is essential for interpreting vaccine data correctly.

Efficacy measures how well a vaccine works under ideal, controlled conditions — in a randomized clinical trial. Effectiveness measures how well it works in the real world, where conditions are messier and populations are more diverse.

Both numbers are valuable. Efficacy tells us the biological effect of the vaccine. Effectiveness tells us the practical public health impact.

Vaccine Efficacy

Efficacy is measured in randomized controlled trials (RCTs) — the gold standard of clinical research. In a vaccine trial:

The formula:

Vaccine Efficacy = (Risk in unvaccinated − Risk in vaccinated) ÷ Risk in unvaccinated

For example: if 1% of unvaccinated participants develop disease and 0.1% of vaccinated participants develop disease, efficacy = (1% − 0.1%) ÷ 1% = 90%. This means the vaccine reduced the risk of disease by 90% under trial conditions.

Vaccine Effectiveness

Effectiveness measures how well a vaccine works in routine, real-world conditions. Unlike trials, effectiveness studies use observational data — looking at health outcomes as they happen across general populations.

Why Effectiveness Usually Differs from Efficacy

Population Differences

Trials often exclude elderly, immunocompromised, or those with comorbidities — the very people who may respond less strongly to vaccines and who represent a large share of real-world vaccine recipients.

Less Strict Protocols

Real-world vaccination may not follow the exact schedule, storage conditions, or dosing as in trials, which can affect the immune response produced.

Waning Immunity

Effectiveness studies measure outcomes over longer periods, during which vaccine-induced protection may decline for some diseases and vaccine types.

Variant Mismatch

New pathogen variants may emerge after vaccine development, reducing effectiveness against infection while often maintaining stronger protection against severe disease.

Behavioral Differences

Trial participants may differ from the general population in health-seeking behavior, use of other interventions, or underlying health behaviors.

Endpoint Definitions

Real-world studies may measure different outcomes than trials — for example, hospitalization instead of any symptomatic infection — making direct comparisons complex.

Putting It Together: COVID-19 Example

The distinction between efficacy and effectiveness became highly visible during the COVID-19 pandemic. Here's how it played out, with sources for each figure:

Phase III Trial Results (Efficacy): The original Pfizer-BioNTech and Moderna mRNA COVID-19 vaccines showed approximately 95% and 94% efficacy respectively against symptomatic infection in trials conducted in 2020 (Polack et al., 2020; Baden et al., 2021).

Real-World Results (Effectiveness): As the pandemic evolved, real-world effectiveness shifted:

  • Against Delta variant: approximately 80–90% effectiveness against symptomatic infection (Lopez Bernal et al., 2021)
  • Against Omicron variant: approximately 40–65% effectiveness against symptomatic infection, though estimates varied significantly by study and timing (Andrews et al., 2022)
  • Against severe disease, hospitalization, and death: effectiveness remained substantially higher than against mild infection — 70–90% in many studies even when infection protection declined (CDC, 2022)

This was not because the vaccines changed, but because: the virus evolved into variants that partially escaped immune protection; immunity waned over time; and real-world conditions differed from trial protocols. The gap between infection protection and severe disease protection reflects the immune system's multiple layers of defense.

Key Takeaways

Sources & Citations

Polack FP, et al. "Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine." New England Journal of Medicine. 2020;383:2603-2615. https://doi.org/10.1056/NEJMoa2034577

Baden LR, et al. "Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine." New England Journal of Medicine. 2021;384:403-416. https://doi.org/10.1056/NEJMoa2035389

Lopez Bernal J, et al. "Effectiveness of Covid-19 Vaccines against the B.1.617.2 (Delta) Variant." New England Journal of Medicine. 2021;385:585-594. https://doi.org/10.1056/NEJMoa2108891

Andrews N, et al. "Covid-19 Vaccine Effectiveness against the Omicron (B.1.1.529) Variant." New England Journal of Medicine. 2022;386:1532-1546. https://doi.org/10.1056/NEJMoa2119451

CDC. "Vaccine Effectiveness: How Well Do Flu Vaccines Work?" Centers for Disease Control and Prevention. https://www.cdc.gov/flu/vaccines-work/vaccineeffect.htm

Halloran ME, Longini IM Jr, Struchiner CJ. Design and Analysis of Vaccine Studies. Springer; 2010. https://doi.org/10.1007/978-0-387-68636-3

WHO. "Vaccine Efficacy, Effectiveness and Protection." World Health Organization. 2021. https://www.who.int/news-room/feature-stories/detail/vaccine-efficacy-effectiveness-and-protection