Figures
Abstract
Background
Four cluster randomized controlled trials (cRCTs) conducted in long-term care facilities (LTCFs) have reported reductions in patient risk through increased healthcare worker (HCW) influenza vaccination. This evidence has led to expansive policies of enforcement that include all staff of acute care hospitals and other healthcare settings beyond LTCFs. We critique and quantify the cRCT evidence for indirect patient benefit underpinning policies of mandatory HCW influenza vaccination.
Methods
Plausibility of the four cRCT findings attributing indirect patient benefits to HCW influenza vaccination was assessed by comparing percentage reductions in patient risk reported by the cRCTs to predicted values. Plausibly predicted values were derived according to the basic mathematical principle of dilution, taking into account HCW influenza vaccine coverage and the specificity of patient outcomes for influenza. Accordingly, predicted values were calculated as a function of relevant compound probabilities including vaccine efficacy (ranging 40–60% in HCWs and favourably assuming the same indirect protection conferred through them to patients) × change in proportionate HCW influenza vaccine coverage (as reported by each cRCT) × percentage of a given patient outcome (e.g. influenza-like illness (ILI) or all-cause mortality) plausibly due to influenza virus. The number needed to vaccinate (NNV) for HCWs to indirectly prevent patient death was recalibrated based on real patient data of hospital-acquired influenza, with adjustment for potential under-detection (5.2-fold), and using favourable assumptions of HCW-attributable risk (ranging 60–80%).
Results
In attributing patient benefit to increased HCW influenza vaccine coverage, each cRCT was found to violate the basic mathematical principle of dilution by reporting greater percentage reductions with less influenza-specific patient outcomes (i.e., all-cause mortality > ILI > laboratory-confirmed influenza) and/or patient mortality reductions exceeding even favourably-derived predicted values by at least 6- to 15-fold. If extrapolated to all LTCF and hospital staff in the United States, the prior cRCT-claimed NNV of 8 would implausibly mean >200,000 and >675,000 patient deaths, respectively, could be prevented annually by HCW influenza vaccination, inconceivably exceeding total US population mortality estimates due to seasonal influenza each year, or during the 1918 pandemic, respectively. More realistic recalibration based on actual patient data instead shows that at least 6000 to 32,000 hospital workers would need to be vaccinated before a single patient death could potentially be averted.
Conclusions
The four cRCTs underpinning policies of enforced HCW influenza vaccination attribute implausibly large reductions in patient risk to HCW vaccination, casting serious doubts on their validity. The impression that unvaccinated HCWs place their patients at great influenza peril is exaggerated. Instead, the HCW-attributable risk and vaccine-preventable fraction both remain unknown and the NNV to achieve patient benefit still requires better understanding. Although current scientific data are inadequate to support the ethical implementation of enforced HCW influenza vaccination, they do not refute approaches to support voluntary vaccination or other more broadly protective practices, such as staying home or masking when acutely ill.
Citation: De Serres G, Skowronski DM, Ward BJ, Gardam M, Lemieux C, Yassi A, et al. (2017) Influenza Vaccination of Healthcare Workers: Critical Analysis of the Evidence for Patient Benefit Underpinning Policies of Enforcement. PLoS ONE 12(1): e0163586. https://doi.org/10.1371/journal.pone.0163586
Editor: Lamberto Manzoli, University of Chieti, ITALY
Received: January 6, 2016; Accepted: August 27, 2016; Published: January 27, 2017
Copyright: © 2017 De Serres et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper.
Funding: Funding support in the form of wages for Dr. Gaston De Serres was provided foremost by the Quebec Public Health Institute (Institut national de santé publique du Québec), Québec, Canada and in part by the Ontario Nurses’ Association. Co-authors were supported by their respective institutions and no other external funding was provided. These funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: GDS, MG, CL and AY have been engaged (with or without remuneration) to provide expert testimony during legal challenges against enforced healthcare workers influenza vaccination policies in Canada. GDS has received grants from GSK and Pfeizer and travel reimbursement to attend an ad hoc advisory board meeting of GSK. BJW has participated in clinical trials and ad hoc advisory boards with several vaccine manufacturers as well as receiving occasional speaker's fees. He has held or holds research grants with GSK and Medicago. He is medical officer for Medicago Inc. and has served as an expert witness for US and Quebec vaccine injury compensation programs. MK has received research grants from Roche, Merck, Siemens, Hologic, and Boerhinger Ingelheim for unrelated studies. FC has received personal fees from AstraZeneca and GlaxoSmithKline for consultancy on influenza epidemiology outside the submitted work. DMS, DMP, ML and PC have no competing interests to report. There are no patents, products in development or marketed products to declare. This does not alter the authors adherence to PLOS ONE policies on sharing data and materials.
Introduction
Annual influenza vaccination for health care workers (HCWs) is widely endorsed [1–5] and increasingly enforced on the basis that it will reduce influenza-associated morbidity and mortality in patients [6–10]. Four cluster randomized controlled trials (cRCTs), conducted exclusively in long-term care facilities (LTCFs) and nursing homes have specifically assessed indirect patient benefits from HCW influenza vaccination and have been most often cited in the support of mandatory HCW policies [11–14]. Two pivotal systematic reviews and meta-analyses have been published summarizing and pooling these four cRCT findings, but reached different conclusions about the strength of that evidence [15–18]. Whereas the review conducted by investigators of the United States Centers for Disease Control and Prevention (US CDC) [15] based on the GRADE framework characterized the overall quality of evidence as moderate [19], the Cochrane review concluded that the evidence was insufficient to support HCW influenza vaccination as an approach to reduce patient risk [16–18]. Such uncertainty in the quality of the evidence warrants closer examination. This is particularly important given that compulsory or coercive (e.g. vaccinate-or-mask) policies have been extrapolated in some jurisdictions to not only include HCWs providing direct patient care in LTCFs, but also to include all staff in acute-care hospitals and other healthcare settings [6–10].
The ethical premise for mandatory HCW influenza vaccination critically hinges upon the valid demonstration of patient benefit substantial enough to justify infringement of the personal rights of HCWs who would otherwise choose not to receive influenza vaccine each year. In this re-analysis, we undertake critique and quantification of the evidence for indirect patient benefit, translating methodological considerations in a way not previously made plain or accessible. In particular, our critique evaluates the plausibility of cRCT conclusions by assessing consistency with the natural boundaries for indirect vaccine effects set by the mathematical principle of dilution, taking into account involved compound probabilities. In addition, to gauge the potential impact of enforced HCW vaccination if further extended to all acute care hospitals, we quantify the number needed to vaccinate (NNV) to potentially prevent a single patient death in the hospital setting—a core epidemiological concept that is critical to policy-makers but that has not previously been addressed in the scientific literature.
Methods
a. Core concepts: vaccine efficacy (VE), outcome specificity, and the principle of dilution
Vaccine efficacy (VE) is the percentage reduction in a particular disease outcome in vaccinated compared to unvaccinated individuals. For example, a VE of 60% means that vaccinated people have a 60% reduction in their risk of a given outcome compared to unvaccinated people.
Studies of VE may include a variety of outcomes for which prevention may be desirable. For example, for influenza, preventable outcomes of interest may include laboratory-confirmed influenza infection, or clinical influenza-like illness (ILI), or hospitalization or death. Illness that is laboratory-confirmed for influenza using molecular diagnostic tools (such as polymerase chain reaction (PCR)) is the most specific study outcome and virtually 100% of cases defined this way are actually related to the influenza virus (i.e. the positive predictive value for influenza in PCR-positive cases is nearly 100%). But it may be challenging to study laboratory-confirmed outcomes and so other surrogate clinical outcomes are sometimes used in VE studies. For example, ILI is typically defined as a clinical syndrome consisting of some constellation of respiratory and systemic symptoms, such as cough and fever, but these are not just features of influenza. ILI can be caused by multiple other non-influenza respiratory pathogens against which influenza vaccine has no effect. When assessing the benefits of influenza vaccine, these other causes of ILI dilute the measured effects. The proportion of seasonal ILI cases actually due to influenza (i.e. positive predictive value for influenza) may vary with defining conditions but is generally <35% [20]. As a more extreme example, the tally of all deaths without laboratory confirmation and regardless of cause (i.e. “all-cause mortality”) includes a vast array of non-influenza etiologies, including not only other infections, but also a multitude of other causes such as cancer, accidents, trauma, cardiovascular events, metabolic conditions, kidney disease etc. The positive predictive value for influenza is even smaller (i.e. more diluted) when using “all-cause mortality” than when using ILI as a study outcome.
It may be of public health interest to assess the benefits of vaccine against the sum total of any ILI or mortality, but to be considered valid these claimed benefits must adhere to the basic principle of dilution. The principle of dilution governs the inverse relationship between percentage reduction in a given outcome, and percentage contribution of non-targeted events to that outcome. Because of dilution effects, the more that non-target conditions contribute to an outcome the lower must be the percentage reduction in that outcome brought about by a targeted intervention. This basic mathematical principle of dilution is universal; it applies not only to vaccination but also to other practical considerations involving percentage reductions. For instance, most shoppers intuitively know that the percentage reduction from a discount coupon will always be greatest in relation to the price of the product featured on the coupon, and lower when calculated in relation to their entire shopping bill that includes other non-applicable items. This commonplace example is shown in the Supporting Information to reinforce the general truism of the basic principle of dilution (S1 Appendix).
Influenza vaccination targets influenza virus and is ineffective against non-influenza causes of ILI, hospitalization or death. The mathematical principle of dilution thus requires that the percentage reduction (i.e. VE) must be lower when non-targeted events (i.e. non-influenza etiologies) are included in the study outcome (e.g. ILI or all-cause mortality) than when only the targeted condition (i.e. influenza infection) contributes. In other words, the lower the positive predictive value of an outcome for influenza, the lower must be the measured VE for that outcome. Under valid conditions, VE will always be higher for laboratory-confirmed influenza than for ILI, and higher for ILI than for all-cause mortality. Where this pattern is not observed, claimed benefits of a vaccine cannot be accepted at face value; other explanatory factors must exist.
In fact, the percentage reduction in a given outcome can never exceed the positive predictive value for influenza of that outcome (even with an influenza VE of 100%), and will be even lower when further taking into account vaccine coverage. When comparing two groups or populations in order to attribute vaccine benefits, the percentage reduction in a given outcome will be proportional to the absolute difference (Δ) in vaccine coverage between the two populations compared.
The percentage reduction in a given outcome attributed to change in vaccination coverage can thus be calculated according to the following equation, based upon the principle of dilution: (1)
b. Critique of cRCT evidence for patient benefit
Through Medline search, we confirmed that the US CDC (published 2014 [15]) and Cochrane (published 2010 [16], 2013 [17] and 2016 [18]) reviews captured all RCTs that specifically assessed indirect patient benefit through HCW influenza vaccination. In the four identified cRCTs, multiple patient outcomes were assessed, ranging from most-to-least specific for influenza (e.g. laboratory-confirmed influenza versus ILI versus all-cause mortality). The 2010 [16] and not later versions of the Cochrane review is used here because for the all-cause mortality outcome the 2013 version [17] excluded the Hayward trial [13] and the 2016 version [18] did not present pooled analysis.
Using the principle of dilution to anchor our analyses and interpretations, we assessed the plausibility of the cRCT results in attributing patient disease reductions to HCW influenza vaccination. Where the pattern of highest-to-lowest percentage reduction against most-to-least specific outcomes (i.e. laboratory-confirmed influenza > ILI > all-cause mortality) was not observed, or the percentage reduction was not proportional to the change in vaccine coverage, a de facto flaw in the exclusive attribution of vaccine effects was understood. To then quantify the potential magnitude of discrepancy in attributing patient outcome reductions to HCW vaccination, we derived the ratio of cRCT-reported versus derived-predicted reductions.
Predicted reductions in patient outcomes were derived according to Eq 1. For this analysis, we assumed a VE of 60% as reported in an earlier meta-analysis of RCTs in young adults that used virologically-confirmed influenza outcomes [21]; however, we also explored for VE of 40%. We conservatively assumed that the direct VE in HCWs would provide equivalent indirect protection to patients. We assumed that across the influenza season the percentage of true influenza was 100% in laboratory-confirmed influenza, 35% in ILI (double that reported by Carman and comparable to or higher than other published estimates) [12,20,22] and 10% in all-cause mortality [23]. Finally, we used the absolute difference (Δ) in HCW influenza vaccine coverage between control and intervention sites as reported by each of the cRCTs.
c. Quantification of the HCW NNV to indirectly prevent patient death
NNV is usually defined as (1/(absolute risk reduction)) where the absolute risk reduction is the difference in incidence between unvaccinated and vaccinated individuals.
For the indirect prevention of patient death through HCW influenza vaccination, the absolute risk reduction is the number of patient deaths that are attributable to hospital-acquired influenza and that are also preventable by HCW vaccination, divided by the number of unvaccinated HCWs who may have caused these deaths. The NNV for HCW influenza vaccination to prevent patient death due to hospital-acquired influenza is thus derived as: or:
(2)
To parameterize the NNV estimate for acute care, we used two large recent studies in the US (Jhung et al) [24] and Canada (Taylor et al) [25]. We assumed that 60% of all hospital-acquired influenza was attributable to HCWs, recognizing patients and visitors are other contributing sources [26–28], and again allowed direct VE of 60% in HCWs [21] and the same indirect protection to be conferred through them to patients.
Results
a. Critique of cRCT evidence for patient benefit in LTCFs
i. Evaluation in the context of the principle of dilution.
As shown in Table 1 and Fig 1, Potter [11] and Carman [12] reported greater percentage reductions with less specific patient outcomes (i.e. percentage reduction in all-cause mortality > ILI > laboratory-confirmed influenza), a reversal of the trend required for plausibility according to the principle of dilution. For laboratory-confirmed influenza, the reported percentage reduction was comparable to or lower than predicted (Potter 35% vs 34%, respectively; Carman 19% vs 28%, respectively) (Table 1, Fig 1). In contrast, the reported percentage reductions for non-specific outcomes were several-fold higher than predicted, exceeding values that could be reasonably attributed to vaccine effects in all four cRCTs (Table 1, Fig 1). Compared to predicted values, reported reductions in patient ILI outcomes were 8-fold higher in the Hayward trial during the 2003–04 season [13], 4-fold higher in the Lemaitre trial [14] and 3-fold higher in the trial by Potter [11] (Table 1, Fig 1). For all-cause mortality there was even greater discrepancy; reported percentage reductions attributed to HCW influenza vaccination were 12-15-fold greater-than-predicted for Potter, Carman, and Hayward, and 6-fold greater in the trial by Lemaitre.
Abbreviations: VE = vaccine efficacy; Δ VC = absolute difference in vaccine coverage; LCI = laboratory-confirmed influenza; ILI = influenza-like illness; ACM = all-cause mortality
Reported versus predicted values for percentage reduction in laboratory-confirmed influenza, influenza-like illness and all-cause mortality among long-term care facility residents are displayed for the four cluster randomized controlled trials to assess indirect benefits of increased healthcare worker influenza vaccination, including: (a) Potter et al [11]; (b) Carman et al [12]; (c) Hayward et al [13]; and (d) Lemaitre et al [14]. Assumptions and derivations for predicted values accompany each study panel.
ii. Period of analysis exposes cRCT bias and error.
In two [11,14] of the four cRCTs, the period of analysis preceding influenza onset exposes the presence of a bias that may contribute to the difference between reported and predicted reductions.
Based on community surveillance, Potter reported outbreaks of influenza A and B beginning in calendar week 6 and week 1, respectively, of 1995, which were 100 and 60 days, respectively, into the follow-up period that started November 1st 1994 [11]. More than half of the difference in the cumulative mortality curves between intervention and control sites had already accrued before the onset of influenza activity from day 60—a period before which non-influenza etiologies, not preventable by influenza vaccination, would have been the more likely cause of death. The non-vaccine factors reducing mortality before the onset of influenza likely continued to play a role beyond day 60 to the end of the follow-up period, further compounding the misattribution of mortality reduction to HCW influenza vaccination even during the influenza period.
Lemaitre et al also acknowledged that rates were lower in their intervention group compared to the control group preceding the influenza peak by 9 weeks for all-cause mortality and 4 weeks for ILI, a between-group difference they suggested was likely due to an early peak in circulation of RSV, against which influenza vaccine has no effect [14]. During the more relevant period of actual influenza circulation, rates of all-cause mortality and ILI were similar between the study groups, with no significant reductions based on HCW vaccination and no difference at the peak of influenza transmission.
While there may have been some sporadic influenza activity before the recognized influenza period, it is unlikely that most of the sustained influenza risk in these studies could have otherwise preceded (or gone completely undetected by) the community surveillance systems. In that context, most of the all-cause mortality difference reported by these studies before the influenza period would have been a measure of bias or confounding, rather than of influenza vaccine effects.
iii. Hayward cRCT [13].
The cRCT by Hayward et al was commissioned as a more definitive test of the earlier trial findings and as such warrants closer scrutiny [13]. This cRCT was conducted over two seasons (2003–4 and 2004–5) during which comparable increments in HCW vaccine coverage between intervention and non-intervention sites (2003–04: Δ = 30% and 2004–05: Δ = 27%) were achieved. Reductions in ILI and all-cause mortality were significant during the 2003–04 season but substantially lower and non-significant the following season (Table 1). The authors mostly dismissed the second-season findings (deriving an NNV based on first season only) on the rationale that influenza activity had been exceptionally low in 2004–05. The CDC and Cochrane authors also excluded the second season from the pooled estimate of their meta-analyses for unclear reasons [15,16]. Since the second season of Hayward has not been influential on policy decisions we have also not considered it in the current analysis, although it is worth noting that surveillance reports show the 2004–05 season to have been comparable to and more representative of most other prior seasons since 2000 [29–31].
Hayward et al stratified their analyses based on influenza and no-influenza periods with significant benefits during the 2003–04 season reported only during the influenza period. The 30% absolute difference (Δ) in vaccine coverage in HCWs may have potentially improved protection against laboratory-confirmed influenza infection in otherwise susceptible HCWs by up to just 18% (i.e. Eq 1: 60% X 100% X 30%). Similarly based on Eq 1, we may have expected corresponding reductions of up to 6% in ILI and 2% in all-cause mortality given proportionate contributions of influenza to these less specific outcomes (35% and 10%, respectively) and their dilution otherwise by non-vaccine-preventable etiologies (Table 1, Fig 1). Instead, the authors reported 50% reduction in ILI and 27% reduction in all-cause mortality (Table 1, Fig 1). Even on the basis of predicted reductions that are optimistic, approximately 90% of the reductions in patient disease reported by the Hayward cRCT cannot be plausibly explained by HCW vaccination, as shown below: (3)
(4)
Even under the further extreme assumption that as much as 50% of ILI and 30% of all-cause mortality might have been due to influenza across the ~2.5 month follow-up period, the HCW vaccine intervention could have only reduced these outcomes by 9% and 5%, respectively (five times lower than reported by Hayward et al), leaving 80% or more of the reported reductions still unexplained by HCW vaccination. Such large reductions in patient risk attributed to HCW influenza vaccination are even more unbelievable when considering that the 2003–04 season was a notorious vaccine mismatch season during which the A(H3N2) variant A/Fujian/411/2002 dominated and was antigenically distinct from the A/Moscow/10/1999-like vaccine strain for which VE <60% was then considerably more likely [32].
The ILI case definition used by Hayward et al was adjusted to be more sensitive in identifying as many cases as possible among elderly patients [13]. The authors allowed any respiratory symptom or sign in the context of a new or worsening condition of any kind to be counted as ILI. Ultimately, there were 300 ILI cases in control homes and 142 in intervention homes. There were 19 and 13 deaths, respectively associated with their broad ILI definition, providing respective ILI case fatalities of 6% (19/300) and 9% (13/142). The total number of residents (denominator) was similar in the intervention and control homes, allowing focus on numerator tallies alone. As shown in Table 2 adapted from the Hayward trial, the number of all-cause deaths in the control homes was 203 and in the intervention homes was 140, an excess of 63 all-cause deaths in the control homes of which just 6 (i.e. 19–13) were associated with ILI.
If all of the excess all-cause mortality in the control homes was due to influenza as assumed by the authors, then it would also require that 90% (i.e. 57/63) of the influenza deaths occurred in the absence of ILI—i.e. without any respiratory symptom or sign as defined by the enhanced surveillance protocol for this study. Unless there was profoundly poor compliance with the study protocol in reporting secondary outcomes of ILI or ILI with death, there must have instead been a serious misattribution of patient mortality to influenza to explain these findings. Although ILI and mortality surveillance were based on weekly aggregate reporting and could not be individually-linked, the aggregate ILI fatalities in the Hayward study were similar to other findings [25] and as such, under-reporting of ILI (and notably in relation to deaths) does not appear to have been a particular issue. While influenza in older people can occur without fever, to accept that nearly all (i.e. 90%) influenza infections in the elderly lack any respiratory feature, despite almost exclusive replication in the respiratory epithelium, would otherwise require a paradigm shift in the current understanding of this disease. Either explanation for these paradoxical findings reinforces concerns about the reliability and validity of the study’s conclusions, particularly in relation to policies to coerce or compel HCW influenza vaccination.
b. HCW NNV to indirectly prevent patient death
In their conclusions, Hayward et al summarized that just eight HCWs would need to be vaccinated to prevent one patient death (i.e. number-needed-to-vaccinate [NNV] = 8) and further proposed in their concluding statements that their results were likely to be generalizable to other care homes and acute hospital settings, in particular geriatric and rehabilitation wards [13].
To put this summary value and conclusion into context, if actually generalized to the ~1.7 million HCWs in LTCFs in the US, Hayward’s estimate of the NNV would mean that 212,500 patient deaths in LTCFs could be prevented every year through HCW vaccination (i.e. 1.7 million/8), an estimate that substantially exceeds the total number of influenza deaths estimated annually for the whole US population (3000–49,000) [33,34]. If further extrapolated to the ~5.5 million hospital workers in the US it would mean that an additional 687,500 patient deaths could be prevented each year—more even than the 675,000 influenza deaths estimated to have occurred in the whole of the US during the 1918 Spanish influenza pandemic [35,36]. Some attempt at recalibration of this NNV is clearly warranted but, surprisingly, has not previously been proposed in the scientific literature.
To re-assess the NNV for acute care we therefore drew on two networks that have published estimates of hospital-acquired influenza—one in the US and one in Canada. The US FluSurv-Net reported by Jhung et al was conducted during the 2010–11 season and covers a population under surveillance of 29 million (~10% of the US population) [24]. We estimated the number of associated HCWs to be 10% of the 5.5 million hospital workers in the US (i.e. 550,000) and assumed 70% vaccine coverage, as documented elsewhere for US hospitals [37]. The study identified 6,171 laboratory-confirmed-influenza cases of which 172 (2.8%) were hospital-acquired and 27 (16%) died (Table 3).
The Canadian network reported by Taylor et al consisted of 54 hospitals that conducted surveillance for hospital-acquired influenza across six seasons (2006–2012), with a variable number of participating hospitals each year [25]. The 2010–11 season had the highest number of hospital-acquired influenza cases and for consistency with the Jhung study, we focused analysis on that season. Together this Canadian hospital-based surveillance network comprises 35% of all hospital bed-days in Canada. There are ~500,000 hospital workers in total in Canada. During the 2010–11 season, 35 (65%) of the 54 hospitals in the network contributed to influenza surveillance activities (personal communication, G. Taylor and D Gravel) [25]. Assuming a 1:1 ratio of HCWs to hospital bed-days, we estimated the number of HCWs employed across the network during the 2010–11 season to be 35% X 65% of the 500,000 hospital workers in Canada (i.e. ~113,000). We further assumed that HCW influenza vaccine coverage across the network was 40%, as documented elsewhere for Canadian hospitals [38,39]. During the 2010–11 season, the study identified 257 healthcare associated influenza cases of which 32% (82) were acquired in hospital (versus long-term care). Applying the 8.9% influenza-associated case fatality reported overall by Taylor to the 2010–11 season, yields 7 hospital-acquired influenza deaths identified across the 35 participating hospitals that season (Table 3).
Of these 27 US and 7 Canadian deaths in 2010–11, we estimated that 8 and 4 respectively were attributed to unvaccinated HCWs and 5 and 2 respectively could be prevented through HCW vaccination, assuming HCW-attributable risk and vaccine efficacy each of 60% (Table 3).
Based on these data, the number of hospital staff that would need to be vaccinated (NNV) to prevent a single patient death was similar in the Canadian and US settings at 32,688 and 32,819, respectively (Table 3). The NNV would be higher with lower VE or lower HCW-attributable risk. If instead of 60%, VE in HCWs was assumed to be 40% or less as reported by several studies in Canada and the US for the 2010–11 season [40–42], the NNV would increase to at least 54,193 and 61,111, respectively. Even allowing that as much as 80% of hospital-acquired influenza deaths were due to HCWs, the NNV in hospitals would still range between 24,516 (Canadian network) and 24,614 (US network) for a VE of 60%, and between 40,645 and 45,833 for a VE of 40%. Applying the highest adjustment factor (5.2) for under-detection of influenza reported by Reed et al [43] for elderly patients admitted to the US hospital network in 2010–11, the NNV would still range between 6286 and 6311 for a VE of 60% and between 10,241 and 11,752 for a VE of 40%.
Discussion
Policies of enforced HCW influenza vaccination are gaining popularity in North America, bolstered primarily by four cRCTs broadly interpreted to prove substantial patient benefit. Our analyses, however, show that these policies lack a solid empirical underpinning. Although RCTs represent the gold-standard study design for assessing interventions, unblinded cRCTs are at greater risk of bias and where their conclusions appear implausible caution is required in their interpretation, particularly if used to guide policies that abrogate individual rights. In that regard, each of the four cRCTs used to champion compulsory HCW influenza vaccination policies reports patient benefits that are mathematically impossible under any reasonable hypothesis of indirect vaccine effect. Whereas it is assumed on the basis of these studies that unvaccinated HCWs place their patients at great influenza peril, we show through detailed critique and numerical recalibration that these impressions are exaggerated.
Eq 1 makes plain the principle of dilution—a relatively simple concept, whose central importance in assessing the reliability of the cRCT evidence seems to have been overlooked by policy analysts to date. We detail it here to expose an obvious incongruity: reported percentage reductions that are greater for less specific outcomes or that exceed increases in vaccine coverage cannot be accepted as valid demonstration of vaccine benefit. To reinforce this understanding, we underscore that other (non-cluster) RCTs that have explicitly reported the effects of dilution have shown its substantial impact. For example, VE against specific laboratory-confirmed influenza vs. non-specific ILI in RCTs involving HIV-negative pregnant mothers was 50% vs. 4% [44]; in HIV-positive adults was 75% vs. -7% [45]; in adults receiving live attenuated influenza vaccine was 42% vs. 4% [46]; and in children receiving quadrivalent vaccine was 59% vs. -3% [47]. In each of these real RCT examples, VE estimates for ILI were at least 10–12 times lower than for laboratory-confirmed influenza. These substantial differences in RCT-derived VE for laboratory-confirmed influenza vs. ILI are in the context of comparing wholly vaccinated to wholly unvaccinated participants and do not require the further downward adjustment associated with partial vaccine coverage as in the cRCTs to assess increased but incomplete HCW vaccine coverage between intervention and control sites. To maximize the predicted benefit to patients we assumed that 35% of ILI could be attributed to influenza, resulting in a VE against ILI that was just three times (not as much as 10–12 times) lower than for laboratory-confirmed influenza. In other words, in our optimistic projections we have substantially under-estimated dilution effects in favour of over-estimating potential patient benefits defined by non-specific outcomes. Even so, the cRCTs to assess indirect patient benefit through increased HCW vaccination have reported estimates that exceed by several-fold even our most optimistic projections of plausibility and are thoroughly inconsistent with the principle of dilution.
Although the derivations we present based on Eq 1 may seem simplistic, they should be understood to have a priori over-estimated potential patient benefit; any other conditional probabilities (decimal multipliers) included as “chain of event” transmission parameters within a more complex model would serve only to further reduce the patient benefits we have optimistically projected as attributable to HCW vaccination. This is shown in the dynamical simulations of van den Dool et al [48,49] incorporating additional parameters beyond direct VE and vaccine coverage, such as the likelihoods of close or casual contacts between different potential sources (HCWs or patients) involved in successful transmission. In both the nursing home and hospital ward simulations by van den Dool et al [48,49], assuming a direct VE of 73% in HCWs, authors showed in their dynamical models that increasing HCW vaccine coverage from zero to 100% reduced the influenza attack rate in patients by about 40%, an indirect VE that is a little more than half the direct VE in HCWs [48,49]. In all of the acute care simulations that varied other parameters but not the direct VE, vaccinating all HCWs gave lower indirect protection to patients (30–50%) than direct protection (73%) to HCWs [48].
The simulations by van den Dool et al are based on small facility settings (e.g. 30-bed nursing home, 24-bed hospital ward) and result in average HCW infection rates (absent vaccination) of 50% in the hospital scenario—more than double the highest (serologically-defined) estimate measured empirically by Elder et al among unvaccinated hospital workers [50], and more than 50 times greater than the rate of PCR-confirmed symptomatic influenza identified among hospital workers during a three-year active surveillance initiative [51,52]. It is difficult to conceive of one in two HCWs becoming infected each year and by furthermore assuming uniform transmission risks whether from symptomatic or asymptomatic infection and that all infected HCWs continued to work and took no other precautions throughout their full infectious period, the van den Dool modeling likely also over-estimates the burden to patients [48,49]. Accordingly, the modeling findings by van den Dool et al should not be interpreted or extrapolated at face value but do nevertheless reinforce the important general message that incorporation of additional multiplicative probabilities reduces indirect VE compared to direct VE. With our assumption that indirect VE in patients is instead equivalent to direct VE in HCWs, our simple model will have therefore prominently over-estimated predicted patient benefits more so even because the direct VE of 60% [21] we have assumed is itself likely an over-estimate. Several studies [40–42,53–59] and a recently updated meta-analysis [60] report lower direct VE (<40%, on average), even in healthy young adults, against A(H3N2)—the subtype that is responsible for nearly 80% of influenza-related deaths, particularly in the elderly [61].
Despite assumptions designed to substantially over-estimate indirect patient benefit, the four cRCTs reported reductions in all-cause mortality among patients that were 6- to 15-fold greater than predicted (Table 1). Even allowing that as much as 50% of ILI and 30% of all-cause mortality across the season could be due to influenza, we found the reductions reported by Hayward et al [13] to be five times greater than predicted, beyond what could be realistically attributed to HCW vaccination. The lower limits of the 95% confidence intervals in pooled meta-analyses by the US CDC and Cochrane groups also exceeded predicted mortality reductions by 4- to 12-fold (Table 1) [15,16]. Even if confidence intervals around the various non-specific outcome estimates overlap, they cannot be cited independently of implausible point estimates. Confidence intervals are meant to reflect variation around true effect size based on the normal fluctuation arising with random sampling, assuming no other bias is influencing the measured effect [62]. However, the confidence interval is directly dependent upon the observed point estimate and the study sample size. To invoke the lower limit of the confidence interval to explain away otherwise implausible point estimates is to conflate the two separate possibilities of random variation (lack of precision, a statistical consideration) and systematic bias (lack of validity, an epidemiological consideration). When the point estimate is not valid (e.g. when there are concerning epidemiological signals of implausibility or bias), the confidence interval directly dependent upon it is also not valid. Random error may add to systematic error, but it provides no defense against an apparent lack of validity in a study’s main finding.
Cluster RCTs are at greater risk of bias to explain findings than trials with individual randomization. As highlighted by the Cochrane group, the cRCTs showing patient benefit through increased HCW vaccination in LTCFs suffered multiple methodological issues including failure to conceal group allocation (i.e. blinding), poor performance in increasing HCW vaccine coverage, insufficient power to assess specific influenza outcomes and the possible influence of selection bias [16]. None of the trials provided information on other co-interventions such as hand washing, quarantining, or requesting/requiring HCWs with ILI to stay home or mask. Education and other promotional efforts delivered as part of a program to increase voluntary vaccination within intervention sites may have simultaneously enhanced awareness of and compliance with these other more broadly protective practices, which may have then confounded or compounded the vaccine effects.
The effects of these non-vaccine factors may be particularly evident in the chosen analysis periods prior to influenza circulation in at least two of the four cRCTs. HCW influenza vaccination cannot prevent ILI or death in patients for which influenza is not somehow implicated and therefore cannot account for differences between intervention and control groups before the period of influenza virus circulation. It is then likely that the confounding factors or co-interventions driving those pre-existing differences between groups had persistent effects through the rest of the study period. This type of confounding in reporting indirect VE against non-specific patient outcomes is reminiscent of the bias previously identified by Jackson et al in reporting significant VE against all-cause pneumonia and mortality in seniors themselves, not only during the influenza period, but also before and after [63]. To further investigate and quantify the impact of this potential bias, future trials should include control outcomes that are clearly not due to influenza and are not preventable by vaccine, and should place greater emphasis on laboratory confirmation when assigning influenza attribution. In the meantime, it should be well understood that HCW vaccination could have only accounted for a small but unknown proportion of the total patient benefits reported by the four cRCTs; most of the observed reductions in non-specific patient outcomes must have instead been due to accompanying non-vaccine protective measures or otherwise reflect unrecognized biases.
Supporters of compulsory policies have cited other studies, including an additional cRCT conducted by Riphagen-Dalhuisen et al in acute care facilities [64]. The main objective of that trial, however, was to assess whether a multi-faceted intervention could increase HCW influenza vaccine coverage. Influenza illness in patients was only assessed as a secondary outcome, without standardized surveillance and based only upon retrospective assessment through computerized discharge notes. The authors reported a significant 50% reduction in the combined outcome of pneumonia and/or influenza among patients in the department of medicine despite just 8% difference in HCW vaccine coverage between the intervention and control groups. As per above, it is not possible to attribute this magnitude of reduction in patient outcome to such minimal change in HCW vaccine uptake, particularly since many pneumonias do not have influenza as a root cause and are not preventable by influenza vaccine, directly or indirectly. The US CDC review also considered four observational studies that have reported indirect patient benefit through HCW influenza vaccination [15]. These observational studies are even more susceptible to bias: each reports percentage reductions (three of four in relation to ILI) that substantially exceed increments in vaccine coverage and are irreconcilable with the principle of dilution [52,65–67].
It is in generalizing beyond the context of voluntary programs in LTCFs to compulsory vaccination of all acute care staff that incongruities in the cRCT evidence become most apparent. A literal extrapolation of the heretofore unchallenged NNV estimate of 8 to all hospital staff leads to the untenable conclusion that unvaccinated HCWs pose a risk to their patients that is tantamount to a 1918-style influenza pandemic every year (within the nosocomial setting alone). To recalibrate these utterly implausible impressions, we drew upon published hospital-based surveillance networks in the US and Canada, focusing on their real data for patient mortality as the outcome most clinically and ethically relevant to the argument for enforced HCW vaccination. This showed that more than 32,000 hospital staff would need to be vaccinated before a single patient death could potentially be averted—4000-fold greater than estimated by Hayward et al [13]. These hospital-based influenza surveillance networks may have missed some hospital-acquired cases but severe cases culminating in death were less likely to have been missed than milder, non-fatal cases or those that were admitted but community-acquired [24, 25]. Even if we nevertheless apply the highest adjustment factor (5.2) for under-detection of influenza as reported by Reed et al for elderly patients admitted to hospital [43] as if it applied to all hospital-acquired influenza, the NNV estimate would still exceed 6000, or about a thousand-fold greater than reported by Hayward et al [13]. We assumed that 60% of hospital-acquired influenza was attributable to HCWs. This may be considered an underestimate of their contribution to original and onward patient transmission but given the role of other influenza sources (i.e. patients and visitors) in the dynamic and porous hospital setting [26–28], our choice more likely represents an over-estimate. We nevertheless explored up to 80% HCW attribution in sensitivity analysis. If policies of enforced HCW vaccination are further expanded beyond hospitals, to also include ambulatory care staff, the NNV to prevent patient death is expected to be even greater, given shorter and less intense contact patterns and risk compared with the care given to bedridden patients. As per above, our assumption that direct VE in HCWs is equivalent to the indirect VE in patients will have prominently under-estimated the NNV. Our NNV estimates should thus be considered minimum values indicative of a more realistic order of magnitude compared to Hayward et al [13], rather than an exact measure of impact. The latter would require actual scientific quantification of the HCW-attributable risk and the fraction indirectly preventable by HCW vaccination, by healthcare setting. However, those values are absent in the published literature for any setting.
Other personal benefits to HCWs or their families, or to their institutions through possible reductions in worker absenteeism, were not factored into our analyses since these do not constitute the main ethical imperative for enforced vaccination of HCWs compared to other citizens or occupational groups. We did not consider the emerging evidence of negative effects from prior vaccination on subsequent influenza vaccine protection or pandemic risk, or the potential liability to institutions or organizations in the event of vaccine-associated harm under compulsory or coercive policies [59,68–77]. Our analyses did not consider the impact of fully implementing other infection control measures (e.g. staying at home when ill, masking when ill, hand hygiene) which, even acknowledging the uncertainty in the success of these interventions [78–81], would reduce the burden of patient deaths preventable through HCW vaccination (further increasing the NNV). We did not address the benefits to patients from more extreme measures such as season-long, asymptomatic mask wearing, offered as the only alternative to vaccine under some HCW policies. If the epidemiologic evidence for indirect patient benefit through HCW influenza vaccination is considered flawed the further hypothesis of indirect patient benefit through obligatory masking of unvaccinated but asymptomatic workers is completely conjectural. There are, of course, other logical fallacies in policies requiring the masking of unvaccinated HCWs throughout the influenza season to protect patients. Given the overall burden of nosocomial respiratory infections, including other serious non-influenza viruses such as RSV or human metapneumovirus, influenza vaccination should be considered insufficient. On average more than 40% of vaccinated HCWs remain vulnerable to seasonal influenza overall [21]—more than 60% in relation to the most dangerous A(H3N2) subtype [60]—and in that context, a coherent prevention policy to reduce risk to patients to the extent possible would dictate the wearing of masks by all HCWs, vaccinated or unvaccinated, for the duration of the winter respiratory season. We are unaware of such extreme policies anywhere to date.
Through this detailed critique and quantification of the evidence we conclude that policies of enforced influenza vaccination of HCWs to reduce patient risk lack a sound empirical basis [82]. In that context, an intuitive sense that there may be some evidence in support of some patient benefit is insufficient scientific basis to ethically override individual HCW rights. While HCWs have an ethical and professional duty not to place their patients at increased risk, so also have advocates for compulsory vaccination a duty to ensure that the evidence they cite is valid and reliable, particularly in the absence of good scientific estimates of patient impact. The diversion of resources from more evidence-based efforts and other important but less tangible costs related to loss of trust and credibility also need to be considered, including the implications for other immunization programs and workplace policies. Although current data are inadequate to support enforced HCW influenza vaccination, they do not refute approaches to support voluntary vaccination or other more broadly protective practices such as staying home or masking when acutely ill.
Supporting Information
S1 Appendix. Practical grocery cart example of the principle of dilution and the relationship between outcome specificity, percentage reduction and coverage.
https://doi.org/10.1371/journal.pone.0163586.s001
(PDF)
Author Contributions
- Conceptualization: GDS DMS.
- Data curation: GDS DMS.
- Formal analysis: GDS DMS.
- Funding acquisition: GDS.
- Investigation: GDS DMS.
- Methodology: GDS DMS.
- Project administration: GDS DMS.
- Resources: GDS DMS.
- Supervision: GDS DMS.
- Validation: GDS DMS BJW MG CL AY DP MK ML PC FC.
- Visualization: GDS DMS.
- Writing – original draft: GDS DMS.
- Writing – review & editing: GDS DMS BJW MG CL AY DP MK ML PC FC.
References
- 1. World Health Organization, Meeting of the Strategic Advisory Group of Experts on immunization, April 2012 –conclusions and recommendations. Wkly Epidemiol Rec 2012; 87: 207–208. Available: http://www.who.int/wer/2012/wer8721.pdf?ua=1.
- 2. Mereckiene J, Cotter S, Nicoll A, Lopalco P, Noori T, Weber JT et al. Seasonal influenza immunization in Europe. Overview of recommendations and vaccination coverage for three seasons: pre-pandemic (2008/09), pandemic (2009/10) and post-pandemic (2010/11). Euro Surveill. 2014; 19(16): pii = 20780. Available: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=20780
- 3. United States Centers for Disease Control and Prevention. Immunization of health-care personnel. Recommendations of the Advisory Committee on Immunization Practices (ACIP). Morb Mortal Wkly Rep 2011; 60: 8–10
- 4. United States Centers for Disease Control and Prevention. Prevention and control of seasonal influenza with vaccines. Recommendations of the Advisory Committee on Immunization Practices—United States, 2013–2014. Morb Mortal Wkly Rep. 2013; 62: 28–29.
- 5. Public Health Agency of Canada. An Advisory Committee Statement (ACS)—National Advisory Committee on Immunization (NACI), Statement on seasonal influenza vaccine for 2014–2015. Available: http://www.phac-aspc.gc.ca/naci-ccni/assets/pdf/flu-grippe-eng.
- 6. New York State Department of Health. Regulation for prevention of influenza transmission by healthcare and residential facility and agency personnel. Prevention of influenza transmission by healthcare and residential facility and agency personnel. Available: https://www.health.ny.gov/diseases/communicable/influenza/seasonal/providers/prevention_of_influenza_transmission/.
- 7. Ksienski DS. Mandatory seasonal influenza vaccination or masking of British Columbia health care workers: Year 1. Can J Pub Health 2014; 105: e312–e316.
- 8. Talbot T, Schaffner W. Commentary: On being the first. Infect Control Hosp Epidemiol 2010; 31: 889–892. pmid:20653446
- 9. Babcock M, Gemeinhart N, Jones M, Dunagan W, Woeltje K. Mandatory influenza vaccination of health care workers: Translating policy to practice. Clin Infect Dis 2010; 50: 459–464. pmid:20064039
- 10. Immunization Action Coalition. Honor roll for patient safety. Available: http://www.immunize.org/laws/influenzahcw.asp.
- 11. Potter J, Stott DJ, Roberts MA, Elder AG, O’Donnel B, Knight PV, et al. Influenza vaccination of health care workers in long-term-care hospitals reduces the mortality of elderly patients. J Infect Dis 1997; 175: 1–6. pmid:8985189
- 12. Carman WF, Elder AG, Wallace LA, McAulay K, Walker A, Murray GD, et al. Effects of influenza vaccination of health-care workers on mortality of elderly people in long-term care: a randomised controlled trial. Lancet 2000; 355: 93–97. pmid:10675165
- 13. Hayward AC, Harling R, Wetten S, Johnson AM, Munro S, Smedley J, et al. Effectiveness of an influenza vaccine program for care home staff to prevent death, morbidity, and health service use among residents: cluster randomised controlled trial. Brit Med J 2006; 333: 1241. pmid:17142257
- 14. Lemaitre M, Meret T, Rothan-Tondeur M, Belmin J, Lejonc JL, Luquel L, et al. Effect of influenza vaccination of nursing home staff on mortality of residents: a cluster-randomized trial. J Am Geriat Soc 2009; 57: 1580–1586. pmid:19682118
- 15. Ahmed F, Lindley MC, Alred N, Weinbaum CM, Grohskopf L. Effect of influenza vaccination of healthcare personnel on morbidity and mortality among patients: systematic review and grading of evidence. Clin Infect Dis 2014; 58: 50–57. pmid:24046301
- 16. Thomas RE, Jefferson T, Lasserson TJ. Influenza vaccination for healthcare workers who work with the elderly. Cochrane Database Sys Rev. 2010 CD005187.
- 17. Thomas RE, Jefferson T, Lasserson TJ. Influenza vaccination for healthcare workers who care for people aged 60 or older living in long-term care institutions. Cochrane Database Sys Rev. 2013 CD005187.
- 18. Thomas RE, Jefferson T, Lasserson TJ. Influenza vaccination for healthcare workers who care for people aged 60 or older living in long-term care institutions. Cochrane Database Sys Rev. 2016 CD005187.
- 19. GRADE working group, Grading of Recommendations, Assessment, Development and Evaluation (GRADE). Available:, http://www.gradeworkinggroup.org.
- 20. Thomas RE. Is influenza-like illness a useful concept and an appropriate test of influenza vaccine effectiveness? Vaccine 2014; 32: 2143–49. pmid:24582634
- 21. Osterholm MT, Kelley NS, Sommer A, Belongia EA. Efficacy and effectiveness of influenza vaccines: a systematic review and meta-analysis. Lancet Infect. Dis 2012; 12: 36–44. pmid:22032844
- 22. Van Buynder PG, Konrad S, Van Buynder JL, Brodkin E, Krajden M, Ramler G, et al. The comparative effectiveness of adjuvanted and unadjuvanted trivalent inactivated influenza vaccine (TIV) in the elderly. Vaccine 2013; 31: 6122–6128. pmid:23933368
- 23. Simonsen L, Taylor RJ, Viboud C, Miller MA, Jackson LA. Mortality benefits of influenza vaccination in elderly people: an ongoing controversy. Lancet Infect Dis. 2007; 7: 658–666. pmid:17897608
- 24. Jhung MA, D'Mello T, Pérez A, Aragon D, Bennett NM, Cooper T, et al. Hospital-onset influenza hospitalizations—United States, 2010–2011. Am J Infect Control 2014; 42: 7–11. pmid:24183534
- 25. Taylor G, Mitchell R, McGeer A, Frenette C, Suh KN, Wong A, et al. Healthcare-associated influenza in Canadian hospitals from 2006 to 2012. Infect Control Hosp Epidemiol 2014; 35: 169–175. pmid:24442080
- 26. Régis C, Escuret V, Barret B, Luxemburger C, Pires-Cronenberger S, Gorain C et al. Influenza like illness syndromes in an acute care geriatric unit in Edouard Herriot Hospital, Lyon, France during 5 seasons. 20th European Congress of Clinical Microbiology and Infectious Diseases, Vienna, Austria, 10–13 April 2010. Abstract P1221. Available: http://www.blackwellpublishing.com/eccmid20/abstract.asp?id=84320
- 27. Régis C, Gorain C, Pires-Cronenberger S, Crozet MN, Moalla M, Najioullah F et al. La grippe nosocomiale chez les adultes à l’hôpital Edouard Herriot, Lyon (France), hivers 2004–2005, 2005–2006 et 2006–2007 / Nosocomial influenza at Edouard Herriot University Hospital, Lyons (France) winters 2004–2005, 2005–2006 and 2006–2007. Bull Epidémiol Hebd 2008; 34: 308–311. Available: http://www.invs.sante.fr/beh/2008/34/#5.
- 28. Cheng VC, Tai JW, Wong LM, Chan JF, Li IW, To KK, et al. Prevention of nosocomial transmission of swine-origin pandemic influenza virus A/H1N1 by infection control bundle. J Hosp Infect 2010; 74: 271–277. pmid:20061056
- 29. Health Protection Agency. HPA national influenza season summary. 2005/2006 season. March 2006. Available: http://webarchive.nationalarchives.gov.uk/20060323120000/http://www.hpa.org.uk/infections/topics_az/influenza/seasonal/activity0506/flureport.htm
- 30. Health Protection Agency. HPA national influenza season summary. 2004/2005 season. May 2005.Available: http://webarchive.nationalarchives.gov.uk/20140714084352/http://www.hpa.org.uk/infections/topics_az/influenza/seasonal/activity0405/text/summary.pdf
- 31. Elliot AJ, Fleming DM. Surveillance of influenza-like illness in England and Wales during 1966–2006. Euro Surveill. 2006; 11(10): pii = 651. Available:http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=651
- 32. Paget WJ, Meerhoff TJ, Meijer A; EISS. Epidemiological and virological assessment of influenza activity in Europe during the 2003–2004 season. Euro Surveill 2005;10:107–11. pmid:15879646
- 33. American Health Care association. The state long-term health care sector: characteristics, utilization and government funding: 2011 update, 6 September 2011; http://www.ahcancal.org/research_data/trends_statistics/Documents/ST_rpt_STStats2011_20110906_FINAL_web.pdf
- 34. United States Centers for Disease Control and Prevention. Estimating seasonal influenza-associated deaths in the United States: CDC study confirms variability of flu, 18 March 2015; http://www.cdc.gov/flu/about/disease/us_flu-related_deaths.htm
- 35. American Hospital Association. AHA Hospital Statistics 2014. Health Forum LLC
- 36. United States Department of Health and Human Services, The Great Pandemic, the United States in 1918–1919; http://www.flu.gov/pandemic/history/1918/the_pandemic
- 37. Influenza Vaccination Coverage Among Health-Care Personnel—United States, 2010–11 Influenza Season. Morb Mortal Wkly Rep. 2011; 60: 1073–1077
- 38. Public Health Toronto. Toronto 2015 Influenza Update, November 13, 2015. Available: http://www.toronto.ca/legdocs/mmis/2015/hl/bgrd/backgroundfile-85948.pdf
- 39. Influenza coverage for BC health care facilities: Influenza immunization coverage among acute care hospital staff, 2014–15. Available: http://www.bccdc.ca/resource-gallery/Documents/Statistics%20and%20Research/Statistics%20and%20Reports/Immunization/Coverage/Acute_HCWs_2014_15_influenza_vaccination_coverage.pdf:
- 40. Ohmit SE, Petrie JG, Malosh RE, Cowling BJ, Thompson MJ, Shay DK et al. Influenza vaccine effectiveness in the community and the household. Clin Infect Dis 2013; 56: 1363–1369. pmid:23413420
- 41. Cowling BJ, Feng S, Finelli L, Steffens A, Fowlkes A. Assessment of influenza vaccine effectiveness in a sentinel surveillance network 2010–13, United States. Vaccine 2016; 34: 61–6. pmid:26611200
- 42. Skowronski DM, Janjua NZ, De Serres G, Winter Al, Dickson JA, Gardy JL, et al. A sentinel platform to evaluate influenza vaccine effectiveness and new variant circulation, Canada 2010–2011 season. Clin Infect Dis 2012; 55: 332–342. pmid:22539661
- 43. Reed C, Chaves SS, Daily Kirley P, Emerson R, Aragon D, Hancock EB, et al. Estimating influenza disease burden from population-based surveillance data in the United States. PLoS One. 2015; 10: e0118369. pmid:25738736
- 44. Madhi SA, Cutland CL, Kuwanda L, Weinberg A, Hugo A, Jones S et al. Influenza vaccination of pregnant women and protection of their infants. N Engl J Med 2014; 371: 918–931. pmid:25184864
- 45. Madhi SA, Maskew M, Koen A, Kuwanda L, Besselaar TG, Naidoo D et al Trivalent inactivated influenza vaccine in African adults infected with human immunodeficient virus: double blind, randomized clinical trial of Efficacy, immunogenicity, and safety. Clin Infect Dis 2011; 52: 128–37. pmid:21148531
- 46. De Villiers PJ, Steele AD, Hiemstra LA, Rappaport R, Dunning AJ, Gruber WC et al. Efficacy and safety of a live attenuated influenza vaccine in adults 60 years of age and older. Vaccine 2009; 28: 228–234. pmid:19796721
- 47. Jain VK, Rivera L, Zaman K, Espos RA, Sirivichayakul C, Quiambao BP et al. Vaccine for prevention of mild and moderate-to-severe influenza in children. N Engl J Med 2013; 369: 2481–91. pmid:24328444
- 48. van den Dool C, Bonten MJ, Hak E, Wallinga J. Modeling the effects of influenza vaccination of health care workers in hospital departments. Vaccine 2009; 27: 6261–6267. pmid:19686690
- 49. van den Dool C, Bonten MJ, Hak E, Heijne JC, Wallinga J. The effects of influenza vaccination of health care workers in nursing homes: insights from a mathematical model. PLoS Med. 2008; 5: e200. pmid:18959470
- 50. Elder AG, O’Donnell B, McCruden EAB, Symington IS, Carman WF. Incidence and recall of influenza in a cohort of Glasgow healthcare workers during the 1993–4 epidemic: results of serum testing and questionnaire. Brit Med J 1996; 313: 1241–1242. pmid:8939114
- 51. Vanhems P Voirin N, Roche S Escuret V, Regis C, Gorain C et al. Risk of influenza-like illness in an acute health care setting during community influenza epidemics in 2004–2005, 2005–2006, and 2006–2007 Arch Intern Med. 2011; 171: 151–157. pmid:21263105
- 52. Bénet T, Régis C, Voirin N, Robert O, Lina B, Cronenberger S, et al. Influenza vaccination of healthcare workers in acute-care hospitals: a case-control study of its effect on hospital-acquired influenza among patients. BMC Infectious Diseases 2012; 12: 30. Available: http://www.biomedcentral.com/1471-2334/12/30 pmid:22292886
- 53. Pebody RG, Andrews N, McMenamin J, Durnall H, Ellis J, Thompson CI, et al. Vaccine effectiveness of 2011/12 trivalent seasonal influenza vaccine in preventing laboratory-confirmed influenza in primary care in the United Kingdom: evidence of waning intra-seasonal protection. Euro Surveill. 2013; 18(5): pii = 20389. Available: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=20389
- 54. McLean HQ, Thompson MG, Sundaram ME, Kieke BA, Gaglani M, Murthy K, et al. Influenza vaccine effectiveness in the United States during 2012–2013: Variable protection by age and virus type. J Infect Dis 2015; 211: 1529–1540. pmid:25406334
- 55. Andrews N, McMenamin J, Durnall H, Ellis J, Lackenby A, Robertson C, et al. Effectiveness of trivalent seasonal influenza vaccine in preventing laboratory-confirmed influenza in primary care in the United Kingdom: 2012/13 end of season results. Euro Surveill 2014; 19(27): pii = 20851. Available: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=20851
- 56. Flannery B, Clippard J, Zimmerman RK, Nowalk MP, Jackson ML, Jackson LA, et al. Early estimates of seasonal influenza vaccine effectiveness—United States, January 2015.Morb Mortal Wkly Rep. 2015; 64: 10–15.
- 57. Valenciano M, Kissling E, Reuss A, Valenciano M, Reuss A, Larrauri A, et al. Vaccine effectiveness in preventing laboratory-confirmed influenza in primary care patients in a season of co-circulation of influenza A(H1N1)pdm09, B and drifted A(H3N2), I-MOVE Multicentre Case–Control Study, Europe 2014/15. Euro Surveill. 2016; 21(7): pii = 30139.
- 58. Pebody R, Warburton F, Ellis J, Andrews N, Thompson C, von Wissmann B, et al. Low effectiveness of seasonal influenza vaccine in preventing laboratory-confirmed influenza in primary care in the United Kingdom: 2014/15 mid-season results. Euro Surveill. 2015; 20(5): pii:21025 Available: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=21025
- 59. Skowronski DM, Chambers C, Sabaiduc S, De Serres G, Winter AL, Dickson JA, et al. A perfect storm: Impact of genomic variation and serial vaccination on low influenza vaccine effectiveness during the 2014–15 season. Clin Infect Dis 2016; 63: 21–32. pmid:27025838
- 60. Belongia EA, Simpson MD, King JP, Sundaram ME, Kelley NS, Osterholm MT, et al. Variable influenza vaccine effectiveness by type and subtype: meta-analysis of studies using the test-negative design. a systematic review and meta-analysis of test-negative design studies. Lancet Infect Dis 2016;16:942–951. pmid:27061888
- 61. Thompson WW, Shay DK, Weintraub E, Brammer L, Cox N, Anderson LJ, et al. Mortality associated with influenza and respiratory syncytial virus in the United States. J Am Med Assoc 2003; 289: 179–186.
- 62. Rothman K, Lash TL, Greenland S. Precision and statistics in epidemiologic studies. In: Rothman K, Lash TL, Greenland S, editors. Modern Epidemiology, 3rd Edition. Philadelphia: Lippincott Williams & Wilkins; 2008. pp. 148–67.
- 63. Jackson LA, Jackson ML, Nelson JC, Neuzil KM, Weiss, NS. Evidence of bias in estimates of influenza vaccine effectiveness in seniors. Int J Epidemiol 2006; 35: 337–344. pmid:16368725
- 64. Riphagen-Dalhuisen J, Burgerhof JG, Frijstein G, van der Geest-Blankert AD, Danhof-Pont MB, de Jager HJ, et al. Hospital-based cluster randomised controlled trial to assess effects of a multi-faceted programme on influenza vaccine coverage among hospital healthcare workers and nosocomial influenza in the Netherlands, 2009 to 2011. Euro Surveill 2013; 18(26): 20512. Available http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=20512 pmid:23827527
- 65. Oshitani H, Saito R, Seki N, Tanabe N, Yamazaki O, Hayashi S, et al. Influenza vaccination levels and influenza-like illness in long-term-care facilities for elderly people in Niigata, Japan, during an influenza A (H3N2) epidemic. Infect Control Hosp Epidemiol 2000; 21: 728–730. pmid:11089658
- 66. Enserink R, Meijer A, Dijkstra F, van Benthem B, van der Steen JT, Haenen A, et al. Absence of influenza A(H1N1) during seasonal and pandemic seasons in a sentinel nursing home surveillance network in the Netherlands. J Am Geriatr Soc 2011; 59: 2301–2305. pmid:22091963
- 67. Wendelboe AM, Avery C, Andrade B, Baumbach J, Landen MG. Importance of employee vaccination against influenza in preventing cases in long-term care facilities. Infect Control Hosp Epidemiol 2011; 32: 990–997. pmid:21931249
- 68. Thompson MG, Naleway A, Fry AM, Ball S, Spencer SM, Reynolds S, et al. Effects of repeated annual inactivated influenza vaccination among healthcare personnel on serum hemagglutination inhibition antibody response to A/Perth/16/2009 (H3N2)-like virus during 2010–11. Vaccine 2016; 34: 981–88. pmid:26813801
- 69. Ohmit SE, Petrie JG, Malosh RE, Fry AM, Thompson MG, Monto AS. Influenza vaccine effectiveness in households with children during the 2012–13 season: assessments of prior vaccination and serologic susceptibility. J Infect Dis 2015; 211: 1519–1528. pmid:25416812
- 70. Ohmit SE, Thompson MG, Petrie JG, Thaker SN, Jackson ML, Belongia EA, et al. Influenza vaccine effectiveness in the 2011–2012 season: protection against each circulating virus and the effect of prior vaccination on estimates. Clin Infect Dis 2014; 58: 319–327. pmid:24235265
- 71. McLean HQ, Thompson MG, Sundaram ME, Meece JK, McClure DL, Friedrich TC, et al. Impact of repeated vaccination on vaccine effectiveness against influenza A(H3N2) and B during 8 seasons. Clin. Infect. Dis. 2014; 59: 1375–1385. pmid:25270645
- 72. Valenciano M, Kissling E, Reuss A, Rizzo C, Gherasim A, Horváth JK, et al. Vaccine effectiveness in preventing laboratory-confirmed influenza in primary care patients in a season of co-circulation of influenza A(H1N1)pdm09, B and drifted A(H3N2), I-MOVE multicentre case–control study, Europe 2014/15. Euro Surveill 2016; 21: pii:30139. Available: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=21385
- 73. Carrat F, Lavenu A, Cauchemez S, Deleger S. Repeated influenza vaccination of healthy children and adults: borrow now, pay later? Epidemiol Infect 2006; 134: 63–70. pmid:16409652
- 74. Bodewes R, Kreijtz JHC, Rimmelzwaan GF. Yearly influenza vaccinations: a double-edged sword? Lancet Infect. Dis. 2009; 9: 784–788. pmid:19879807
- 75. Bodewes R, Fraaij PL, Kreijtz JH, Hillaire ML, Vogelzang-van Trierum SE, Nieuwkoop NJ, et al. Annual influenza vaccination affects the development of heterosubtypic immunity. Vaccine 2012; 30: 7407–7410. pmid:22643217
- 76. Skowronski DM, De Serres G, Crowcroft NS, Janjua NZ, Boulianne N, Hottes TS, et al. Association between the 2008–09 seasonal influenza vaccine and pandemic H1N1 illness during spring-summer 2009: Four observational studies from Canada. PLoS Medicine 2010; 7: e1000258
- 77. Skowronski DM, Hamelin ME, De Serres G, Janjua NZ, Li G, Sabaiduc S, et al. Randomized controlled ferret study to assess the direct impact of 2008–09 trivalent inactivated influenza vaccine on A(H1N1)pdm09 disease risk. PLoS ONE 2014; 9: e86555. pmid:24475142
- 78. Kuster SP, Coleman BL, Raboud J, McNeil S, De Serres G, Gubbay J, et al. Risk factors for influenza among health care workers during 2009 pandemic, Toronto, Ontario, Canada. Emerg Infect Dis.2013; 19: 606–615. pmid:23631831
- 79. Turnberg W, Daniell W, Duchin J. Influenza vaccination and sick leave practices and perceptions reported by health care workers in ambulatory care settings. Am J Infect Control 2010; 38: 486–488. pmid:20176412
- 80. Coleman BL, Boggild AK, Drews SJ, Li Y, Low DE, McGeer AJ. Respiratory illnesses in Canadian health care workers: a pilot study of influenza vaccine and oseltamivir prophylaxis during the 2007/2008 influenza season. Influenza Other Resp Viruses 2011; 5: 404–408.
- 81. Ridgway JP, Bartlett AH, Garcia-Houchins S, Cariño S, Enriquez A, Marrs R, et al. Influenza among afebrile and vaccinated healthcare workers. Clin Infect Dis. 2015; 60: 1591–1595. pmid:25733370
- 82. De Serres G, Skowronski DM, Patrick DM. Has the emperor got clothes: scientific evidence and compulsory health care worker influenza immunization. Can Med Assoc J; 2012. Available: http://www.cmaj.ca/content/early/2012/10/29/cmaj.121679.full/reply#cmaj_el_713530