Once again, the flu season is well underway and there is no shortage of commercials, posters, billboards, and doctors telling us to get our flu shot. But is the flu shot really your best shot at preventing the flu? In part 1 of this series on flu prevention I hope to prove to you that the flu vaccine isn’t as impressive as we have been led to believe. In the next post(s) I will discuss what has been proven in the research to naturally prevent the flu.
Flawed Studies and Quality of Evidence
I know you’re not all scientists, but a quick overview of the different forms of scientific research is necessary to fully appreciate the second half of this article. There are many types of research articles out there, and they all have varying degrees of abundance and credibility in the scientific community. For a more detailed description and explanation please see this website. For now all you need to know is that not all research is created equal. The most credible is the meta-analysis, followed by systemic reviews, randomized double-blind placebo studies, then cohort studies. Most of the research on the flu vaccine tends to be cohort studies, which are decent, but can still have a lot of flaws. Randomized control trials eliminate a lot of bias, but may still have issues as we will discuss below. Meta-analyses are considered to be the best because of their rigorous inclusion criteria. These studies compile information from many different articles, and more importantly, only the best articles make it into a meta-analysis.
There are many different issues surrounding the debate of Influenza Vaccine Effectiveness. Issues like cohort (study group) selection bias, non-specific end points, and proper diagnosis of the flu can all stand in the way of an otherwise decent study, and will undoubtedly alter the reported numbers. Because it seems to be the most commonly recognized confounding variable in cohort studies, I will focus briefly on selection bias.
The term “selection bias” is a little bit of a misnomer in this case, because it is seen in cohort studies where the groups are not actually “selected” by the scientists. Rather, these studies follow two groups of people (those who got the vaccine and those who did not) and use the information to estimate how effective the vaccine is. The problem is that it’s nearly impossible to account for other differences between the two groups of people. Factors such as health status, belief or disbelief in vaccine effectiveness, vaccine availability and possibly other unknown variables may all lead to the difference in behavior studied (i.e. the behavior to get the vaccine or to choose to not be vaccinated).
A great deal of skepticism comes from what scientists are calling the “healthy user” effect, which means that the people who are getting vaccinated tend to be healthier, and therefore less likely to get the flu or die from it, than those who do not get the vaccine. This is the image of the ideal patient- someone who listens to their doctor’s advice; whether it be to eat more fiber, exercise, or get a vaccine [4]. One study examined this more in depth and compared several variables between the two groups: age, sex, the log of the insurance risk score, self-assessed health status, and presence of chronic diseases such as diabetes and heart disease. This study found a strong correlation between insurance risk score and self-assessed health status with risk of death, but more importantly they found a strong negative correlation between risk of death (poorer health prior to the flu season) and vaccine coverage. In other words, the more frail people, those who were the most likely to die with or without the flu, were among the least likely to get the flu shot, which skews the results of cohort studies in favor of higher vaccine effectiveness [5].
Aside from the patient’s conscious choice there are also other factors that may create this effect. For example, vaccination is generally not recommended for the extremely frail or immuno-compromised, a group that would be expected to have naturally higher hospitalization and death rates and get lumped into the “unvaccinated” group in a simple cohort study [4]. This is called a “frailty selection” [1]. In placebo controlled studies, it is considered unethical to include high-risk patients in such studies (it’s not cool to give someone a placebo when there is a higher likelihood that the vaccine can save their life), so there is absolutely no high-quality (double blind, placebo controlled) data on the group that is theorized to benefit from vaccination the most [2].
In their paper, Simonsen et al [1] outlined criteria that can be used to detect such bias. The authors note, however, that it is easier to detect bias than to account for it and calculate exactly how much of the results are due to bias.
Seasonality. Think about it, the vaccine’s ability to prevent death should be seasonal- only during the flu season when the virus is actually circulating. If there is a benefit outside of the flu season when the virus is not circulating, this is a strong indicator of a difference between the two groups. Several studies have identified a similar effect on all-cause mortality and hospitalization both during flu season and the off season, indicating an intrinsic difference between the ultimately immunized and non-immunized cohorts [1,6,7]. In 2008 Eurich et al demonstrated a 51% reduction in all cause mortality in pneumonia patients outside the flu season in those who would eventually get the flu shot- a number that is strikingly similar to what other groups report during the flu season [6]. The authors go on to say that this indicates that some or all of the benefit in mortality that is seen in cohort studies is likely due to confounding variables such as the healthy user effect [1].
Vaccine match. Each year the flu virus mutates, creating the supposed need to get the updated yearly flu shot. Each summer and fall, scientists have to try to predict which flu virus will be the predominant strain to make a vaccine against it. Some seasons end up being better matches than others. Such information can be found on the CDC website [8]. Logically, the effectiveness of the vaccine should fluctuate with how well the vaccine is matched to the circulating strain. However, this does not appear to be the case [1,4]. Below is a chart from a 2012 meta-analysis (remember, this means that the studies they use should be the best of the best) that shows how well the vaccine is matched and effectiveness [1]. As you can see, most of the studies from the years that were reported as being mismatched were reported to have the same or better effectiveness than other years. How can that possibly be?
The use of specific end points is important for obtaining accurate, clean results in a study. The more specific your end point (the thing you’re measuring), the better the data you can obtain. Three things that are commonly used to describe flu vaccine effectiveness are all cause mortality, Influenza-like illness (someone says they have the flu), and laboratory confirmed influenza. Since people die from a lot of different things, even during the flu season, we would expect the vaccine’s effectiveness at preventing all-cause mortality to be relatively low. On the flip side, since the specific goal of the vaccine is to prevent the flu (as opposed to Influenza-like illness, which may be the flu or something else), we would expect the vaccine’s effectiveness here to be highest. On the contrary, current cohort evidence shows vaccine effectiveness estimates are highest (and implausibly large) for all-cause mortality and lowest for laboratory confirmed influenza [1].
Several authors have noted that the repeatedly reported 50% effectiveness against all-cause mortality is simply not realistic. The flu only accounts for about 5% of all deaths in the winter, so “that the influenza vaccine can prevent ten times as many deaths as the disease itself causes is not plausible”[1]. Because influenza only contributes a small percentage to all-cause mortality, it cannot be reasonably expected to do more than eliminate this excess. As I discuss below, this turns out to be the result of sloppy math… But there is another point to be made here, first. Several studies in the United States have tried to evaluate whether there has been a decrease in all-cause mortality with increasing vaccination rates. In the last twenty years, since the flu vaccination rate among the elderly has soared from 15% to 65% there has not been a single study that has documented any change in hospital admission rates nor all-cause mortality [6]. Similarly, studies failed to show an increase in mortality during the 1997-98 flu season- a year when the vaccine was completely mismatched with the circulating strain [1].
Math, Statistics, and Interpreting the Numbers
For the time being, let’s put aside all the stuff I just talked about and focus on the numbers- we’ll assume that the data the studies are getting is actually reflective of what the vaccine is doing and that none of it is from confounding variables (big assumptions to make, but stay with me).
There are two ways to discuss the effectiveness or risk of something:
in absolute or relative terms.
100 people get the flu vaccine. Two of them get the flu. 100 other people do not get the vaccine. Four of them get the flu. How effective is the vaccine?
-
Absolute decrease: 2%
-
4 – 2 = 2%
-
-
Relative decrease: 50%
-
4 ÷ 2 = 50%
-
The difference between the two is important to be aware of. This is used in advertising, marketing, and medicine all the time. 50% sounds a LOT better than 2%, so relative difference is often used when the advertiser wants to show you the benefit of something. On the other hand, relative difference is often used when the advertiser is trying to deemphasize something in the data. For example, GSK’s website for Wellbutrin states that 21.1% of Wellbutrin users developed a tremor compared to only 7.6% of placebo users. That’s a 3-fold difference or 300% increase if we speak in relative terms!
Back to the Flu
Vaccine studies generally want to portray the benefit of using their product, so they report effectiveness as a relative %. Let’s look at the numbers from the studies cited by that 2012 meta-analysis again and compare the absolute values to the relative values:
We tend to see things how they relate to ourselves, and we like to apply numbers to real life. The reality is that most people’s odds of getting the flu are already pretty low (2.73% odds without the vaccine if we use the numbers cited above), especially if you take care of yourself. The vaccine then lowers your (average person) risk of getting the flu from 2.73% to 1.18%- which is only an overall difference in risk of 1.55%.
Another way of looking at it is that you need to vaccinate 100 people to prevent ONE set of influenza symptoms [10]. Fireman et al estimate that you would need to vaccinate 4,000 people to prevent one death from the flu in their elderly Kaiser Permanente cohort [5]. This is the group of people theorized to need the vaccine the most and whom make up approximately 90% of influenza related deaths. Let’s not forget that this doesn’t even take into consideration the numerous confounding variables we talked about in the first half of the article, some of which have been shown to either partially or completely explain any benefit the vaccine has been shown to have.
Not only is this misleading to the lay person, but researchers seem to be confused by their own statistics. Recall the authors who said that the 50% reduction in all-cause mortality reported with the vaccine was unrealistic because the flu itself only accounts for about 5% of all winter deaths. Their mistake was in trying to compare apples to oranges (absolute and relative reduction). If they used the numbers from the Fireman et al study, 1 death prevented for every 4,000 vaccinated, that would have made more sense. Of course, this is only mentioned once toward the end of the article- if you read their abstract or their conclusions (let’s face it- that’s what most people read) this group also cites a vaccine effectiveness rate of 47% [5].
Protecting the Vulnerable
Numerous studies have demonstrated that the flu vaccine is least effective in the two groups that are presumed to need it the most- children and the elderly [1,4]. In one trial the researchers estimate that the vaccine effectiveness drops from 75% to 23% between the ages of 65 and 70- a huge and important drop considering that 90% of all influenza deaths are in folks over the age of 65. There were no randomized clinical control trials that were good enough to meet inclusion criteria for the 2012 meta-analysis that showed TIV (trivalent influenza vaccine) effectiveness in people aged 2-17 or over 65 [4]. Only the live attenuated vaccine (LAIV) was shown to have any effect in the 65 years old plus crowd, but that vaccine is not approved for use in adults over 50 in the United States [4]. Perhaps most importantly for seniors, the influenza vaccine has been shown to do nothing to prevent pneumonia. Pneumonia is responsible for the vast majority of influenza-related deaths (approximately 34,000 of the 36,000 deaths (94.4%)) each year [9], so this doesn’t lend much credibility to those estimates on decreasing all-cause mortality.
Some may say that this is why we healthy adults need to get vaccinated- to contain the virus and decrease transmission to vulnerable populations. The same argument is used when discussing vaccine policy in the healthcare setting (doctors and nurses being required to receive the vaccine). It’s difficult to study the transmission of flu in the general population, but there are a limited number of studies on healthcare workers and flu transmission. I think we can use the healthcare setting as a proxy and extrapolate to the greater population- if studies show that vaccinating healthcare workers prevents spreading of the virus then that is likely the case for the general population. From what I’ve read, it appears that there is weak evidence to support this in the healthcare setting. A 2016 review article concluded:
“Offering influenza vaccination to healthcare workers who care for those aged 60 or over in LTCIs may have little or no effect on laboratory-proven influenza (low quality evidence). HCW vaccination programmes probably have a small effect on lower respiratory tract infection (moderate quality evidence), but they may have little or no effect on admission to hospital (low quality evidence). It is unclear what effect vaccination programmes have on death due to lower respiratory tract illness (very low quality evidence) or all cause deaths (very low quality evidence).“
I’d like to mention something I noticed when I was researching this part of the article. At first glance it appears that there are a lot more studies assessing vaccine effectiveness in the healthcare setting than there actually are. However, when you read them the vast majority are papers 1. discussing how to incentivize healthcare workers into getting the vaccine and 2. discussing the rates of non-compliance in various locations. In other words: the number of papers on how to enforce mandatory influenza vaccine policies far outnumber the number of papers assessing whether or not such policies are actually effective.
Closing Thoughts
There are a LOT of things to consider when it comes to this body of research. There are numerous confounding variables that may either partially or completely account for the reported effectiveness of the vaccine, “effectiveness” is inconsistently defined, there is a lack of solid evidence that the vaccine works at all in the populations that supposedly need it the most, there is a lack of evidence that vaccinating healthy adults protects vulnerable populations, and the method of reporting the vaccine effectiveness is misleading at best (statistics). There are other issues that simply could not make it into this post such as the ingredients in the vaccine, side effects, politics, and conflicts of interest with funding sources. Whether you choose to get the vaccine or not is your personal choice, and I understand that there are a lot of factors that do, and should, go into this decision.
Why I am passionate about this topic
-
I’m a big believer in informed consent. You cannot make a truly rational decision for yourself or your children if you don’t know the risks and benefits, or if politics and money have clouded that data. I want to make this as transparent as possible so that each person can make their choice. My biased hope/belief is that if people understood what I do about this vaccine then far fewer would opt to get it.
-
I work with a unique vulnerable population- autoimmune and inflamed people. These people are especially vulnerable to inflammatory and immune stress, which can easily come from a simple vaccine. Yet, they walk among us every day and are also encouraged/pressured into getting the flu vaccine. I want those people to fully understand (again, informed consent) what the vaccine can do to them should they choose to get it.
-
Vaccines aren’t benign and they can cause problems in vulnerable people. Vaccines work by stimulating an immune response, and that can be inflammatory in the wrong person. We need to treat vaccines and drugs with the respect they deserve and use them with caution.
Next in the Flu Series:
Even after all that, we still haven’t gotten to talk about what actually causes the flu. I know what you’re thinking- the influenza virus causes the flu. But the reality is that it is predominantly your health as host that determines whether or not a virus will successfully make you sick. We’ve all lived with someone who was sick but didn’t get sick ourselves. There had to have been a reason why you didn’t get sick, and it wasn’t from lack of exposure. Next time we’re going to talk about what actually causes the flu, as well as what natural things you can do to prevent it. Spoiler alert- a lowly vitamin kicks vaccine butt at preventing the flu.
Wishing you the best of health,
References:
[1] Simonsen, L. Mortality benefits of influenza vaccination in elderly people: an ongoing controversy. Lancet Infectious Dis 2007;7:658-66
[2] Jackson, L. Safety, efficacy and immunogenicity of an inactivated influenza vaccine in healthy adults: a randomized, placebo-controlled trial over two influenza seasons. British Medical Journal Infectious Diseases 2010, 10:71
[3] Lang, P. Effectiveness of influenza vaccine in aging and older adults: comprehensive analysis of the evidence. Clinical Interventions in Aging 2012:7; 55-64
[4] Osterholm, M. Efficacy and effectiveness of influenza vaccines: a systemic review and meta-analysis. Lancet Infectious Disease 2012; 12: 36-44
[5] Fireman, B. Influenza vaccination and mortality: Differentiating vaccine effects from bias. American Journal of Epidemiology 2009;170:650-656
[6] Eurich D. Mortality reduction with influenza vaccine in patients with pneumonia outside “flu” season. American Journal of Respiratory Care Medicine 2008;178:527-533
[7] Hottes, T. Influenza vaccine effectiveness in the elderly based on administrative databases: change in immunization habit as a marker for bias. PLoS ONE 6970: e22618
[8] http://www.cdc.gov/flu/pastseasons/1112season.htm
[9] Jackson. Influenza vaccination and risk of community acquired pneumonia in immunocompetent elderly people: a population-based nested case-control study. 2008;372:398-405
[10] Jefferson, T. Vaccination for preventing influenza in healthy adults. Cochrane database of systemic reviews; 8 article CDOO4879
