Efficacy vs. Effectiveness
Understanding the difference between trial results and real-world performance
These two terms are often confused but measure different things. Understanding the distinction is essential for interpreting vaccine data correctly.
Efficacy measures how well a vaccine works under ideal, controlled conditions — in a randomized clinical trial. Effectiveness measures how well it works in the real world, where conditions are messier and populations are more diverse.
Both numbers are valuable. Efficacy tells us the biological effect of the vaccine. Effectiveness tells us the practical public health impact.
Vaccine Efficacy
Efficacy is measured in randomized controlled trials (RCTs) — the gold standard of clinical research. In a vaccine trial:
- Thousands of volunteers are randomly assigned to receive either the vaccine or a placebo
- Neither participants nor researchers know who gets which (double-blind)
- Participants are followed over time to see who develops the outcome of interest
- Efficacy is calculated as the relative reduction in disease among vaccinated vs. unvaccinated
The formula:
Vaccine Efficacy = (Risk in unvaccinated − Risk in vaccinated) ÷ Risk in unvaccinated
For example: if 1% of unvaccinated participants develop disease and 0.1% of vaccinated participants develop disease, efficacy = (1% − 0.1%) ÷ 1% = 90%. This means the vaccine reduced the risk of disease by 90% under trial conditions.
Vaccine Effectiveness
Effectiveness measures how well a vaccine works in routine, real-world conditions. Unlike trials, effectiveness studies use observational data — looking at health outcomes as they happen across general populations.
Why Effectiveness Usually Differs from Efficacy
Population Differences
Trials often exclude elderly, immunocompromised, or those with comorbidities — the very people who may respond less strongly to vaccines and who represent a large share of real-world vaccine recipients.
Less Strict Protocols
Real-world vaccination may not follow the exact schedule, storage conditions, or dosing as in trials, which can affect the immune response produced.
Waning Immunity
Effectiveness studies measure outcomes over longer periods, during which vaccine-induced protection may decline for some diseases and vaccine types.
Variant Mismatch
New pathogen variants may emerge after vaccine development, reducing effectiveness against infection while often maintaining stronger protection against severe disease.
Behavioral Differences
Trial participants may differ from the general population in health-seeking behavior, use of other interventions, or underlying health behaviors.
Endpoint Definitions
Real-world studies may measure different outcomes than trials — for example, hospitalization instead of any symptomatic infection — making direct comparisons complex.
Putting It Together: COVID-19 Example
The distinction between efficacy and effectiveness became highly visible during the COVID-19 pandemic. Here's how it played out, with sources for each figure:
Phase III Trial Results (Efficacy): The original Pfizer-BioNTech and Moderna mRNA COVID-19 vaccines showed approximately 95% and 94% efficacy respectively against symptomatic infection in trials conducted in 2020 (Polack et al., 2020; Baden et al., 2021).
Real-World Results (Effectiveness): As the pandemic evolved, real-world effectiveness shifted:
- Against Delta variant: approximately 80–90% effectiveness against symptomatic infection (Lopez Bernal et al., 2021)
- Against Omicron variant: approximately 40–65% effectiveness against symptomatic infection, though estimates varied significantly by study and timing (Andrews et al., 2022)
- Against severe disease, hospitalization, and death: effectiveness remained substantially higher than against mild infection — 70–90% in many studies even when infection protection declined (CDC, 2022)
This was not because the vaccines changed, but because: the virus evolved into variants that partially escaped immune protection; immunity waned over time; and real-world conditions differed from trial protocols. The gap between infection protection and severe disease protection reflects the immune system's multiple layers of defense.
Key Takeaways
- A high efficacy number from a trial does not guarantee the same effectiveness in the real world
- Effectiveness against severe disease typically holds up better than effectiveness against mild infection
- Both efficacy and effectiveness figures should always be read alongside their endpoints — what outcome was measured, in which population, and at what time point
- Waning effectiveness over time does not necessarily mean a vaccine has "stopped working" — the degree of waning varies by outcome, pathogen, and individual
Sources & Citations
Polack FP, et al. "Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine." New England Journal of Medicine. 2020;383:2603-2615. https://doi.org/10.1056/NEJMoa2034577
Baden LR, et al. "Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine." New England Journal of Medicine. 2021;384:403-416. https://doi.org/10.1056/NEJMoa2035389
Lopez Bernal J, et al. "Effectiveness of Covid-19 Vaccines against the B.1.617.2 (Delta) Variant." New England Journal of Medicine. 2021;385:585-594. https://doi.org/10.1056/NEJMoa2108891
Andrews N, et al. "Covid-19 Vaccine Effectiveness against the Omicron (B.1.1.529) Variant." New England Journal of Medicine. 2022;386:1532-1546. https://doi.org/10.1056/NEJMoa2119451
CDC. "Vaccine Effectiveness: How Well Do Flu Vaccines Work?" Centers for Disease Control and Prevention. https://www.cdc.gov/flu/vaccines-work/vaccineeffect.htm
Halloran ME, Longini IM Jr, Struchiner CJ. Design and Analysis of Vaccine Studies. Springer; 2010. https://doi.org/10.1007/978-0-387-68636-3
WHO. "Vaccine Efficacy, Effectiveness and Protection." World Health Organization. 2021. https://www.who.int/news-room/feature-stories/detail/vaccine-efficacy-effectiveness-and-protection