In February 2019, I attended a meeting of the Advisory Committee for Immunization Practices (ACIP) for the first time. This post describes my observations from the two-day ACIP meeting process.
Generally, the meeting taught me that the process the committee goes through is highly deliberative, data-intensive, and the committee pays close attention to safety and maximizing benefits. Though no process is perfect, the meeting increased my confidence in the decision-making process behind the vaccines recommendations that apply to my children.
Numerous anti-vaccine group attended added some excitement and some stress, but was, from a standpoint of vaccine policy-making, largely irrelevant.
I am initially, a public administration scholar – I wrote my dissertation on agency accountability, taught the Federal Advisory Committee Act multiple times, and teach almost annually about agency decision making. This made me very interested in the committee’s process. I also knew in advance that there will be – as there has been in several previous meeting – numerous anti-vaccine activists, and was curious to see their interaction with the meeting in reality.
Initially, I thought I would describe in detail what was addressed in the meeting, but I think that would make this post too long. For those who are interested, here is the agenda for the February 2019 ACIP meeting (pdf).
Instead, I will offer my observations about the process. I will mention that the only things voted on in this meeting were related to Japanese encephalitis vaccine and anthrax vaccine. The committee voted to make some changes to the language of the recommendation of the Japanese encephalitis vaccine for travelers to clarify it, but not changes to the actual recommendation, changes to the timeline for adult priming series (the initial vaccine series) from a 28-day interval to an interval that can span 7-28 days, and expanding the age for recommending a booster for children and putting that recommendation on equal footing to the recommendation for an adult booster.
With respect to anthrax vaccines, the committee recommended giving a booster dose to high-risk people (like first responders) who are not currently exposed but may be at risk of exposure, if they want it.
Everything else discussed was informational – some of it as part of the process of preparing for future votes (Like whether to extend the recommendation for HPV vaccines to include those 26-45), some of it as part of ongoing monitoring (like the examination of flu vaccines’ data).
The Advisory Committee for Immunization Practices is an expert committee entrusted with a number of important roles. Besides recommending which vaccines should be on which schedule (children, adults, which groups), and at which ages, the committee, for example, recommends whether to include vaccines in the Vaccines for Children program, a federal benefit program that provides vaccines for children who are not otherwise covered by public or private health insurance.
ACIP meets three times a year for two days. Its fifteen members have a variety of relevant areas of expertise, including, for example, epidemiology, immunology, and infectious diseases. There is also a consumer representative.
Representatives from a range of liaison groups – including medical associations, other agencies (like the department of defense, FDA) and others – also attend the meetings and participate, though they do not vote. The committee works mostly through workgroups made of committee members, government officials, and external experts. Workgroups are usually created for each new vaccine several years before it is cleared by the FDA, as it approaches the end of the clinical trials process. Workgroups also look at existing vaccines and other issues.
As required by the Federal Advisory Committee Act, the committee publishes notices and agenda of meetings. ACIP meetings are also live streamed, and recordings are later available online, as are minutes and slides, at least from recent years. Committee deliberations are, in that sense, extremely transparent, but the data provided in them is the end product of extensive preparatory work by the workgroups – the tip of the iceberg.
ACIP meeting observations – fact-intensive
There is an extensive amount of data provided in the meeting. While it is clearly the summary of much larger amounts of data reviewed by the workgroup, what’s provided in the meeting is almost overwhelming – okay, actually overwhelming. The data is directed at the experts, though there were clear efforts by many of the presenters to make it accessible to lay people.
For example, each presentation about a vaccine started with a review of the basics of the disease, something the experts on the committee and the liaisons were likely well familiar with. In one discussion of safety data, the presenter worked through a definition of terms, including the distinction between an adverse event (a problem reported after a vaccine, which may or may not be caused by it) and an adverse reaction, a problem for which there is strong data indicating a link to the vaccine.
There were many, many graphs summarizing large amounts of data on both safety and effectiveness, and the pace was fast.
The influenza presentation is a good example. It included data about what is going on with influenza – trends, which viruses, medical visits, hospitalizations, and deaths. While this year is nowhere near as bad as last year’s influenza, the preliminary burden estimates were at 17-20 million illnesses, 8.2-9.6 million medical visits, 214-256,000 hospitalizations, 13,600-22,300 deaths.
This was followed by a detailed discussion of vaccine estimates in the United States, generally and across age groups, compared to data from other countries on vaccine estimates (including also data about the live influenza vaccine). Then was a detailed discussion of a large clinical trial of one brand of influenza vaccine – Afluria quadrivalent – in children six to fifty-nine months. Finally, there was a thorough discussion of a multi-year study of spontaneous abortions – miscarriages – after influenza vaccines (which found no increased risk of miscarriages in vaccinated women).
One useful thing that was presented during this session for, apparently, the first time is a graphic display of the benefits of influenza vaccine in the season, with illnesses, hospitalization, and deaths averted, calculated based on a combination of vaccine effectiveness and disease burden. Even with a vaccine effectiveness of around 40%, the benefits were substantial. Influenza vaccines averted 7.1 million cases of the disease, 109,000 hospitalizations, 8,000 deaths in 2017-2018. The presenters said they will provide this information each year, going forward. Good.
Similar large amounts of information were presented in every session.
ACIP meeting observations – close scrutiny of safety
The discussion showed that safety is a priority, taken seriously, closely scrutinized. For example, two of the studies addressing influenza vaccines looked at safety. One looked at the safety of using one brand – Afluria quadrivalent – in children 6-59 months. It discussed solicited reactions, fever, and the 16 serious adverse reactions in the trial, and explained that none were found related to the vaccine.
In the questions, ACIP asked for elaboration about the serious adverse events. The presenter of the study addressed them – I did not have time to write all, but they included: “two febrile seizures. 3 RSV infections, 1 animal bite, 1 broken arm, one gastrointestinal episode.” In the initial discussion, the presenter had already addressed the febrile seizures: “Two of special interest – two simple febrile seizures, one at day 43 and one 103. Neither febrile seizure happened at the 7 days post vaccine – the risk period. Determined unrelated to the vaccine.” (I am using my notes for these quotes since a full video is not yet available, so they may be a short version of the statements).
The second flu discussion was a response to a previous study that found a statistically significant association between flu vaccine and miscarriage in women who have had two H1N1 containing vaccines in a row. That study has been criticized, with experts claiming that the result was pretty clearly not a reflection of a real problem, given previous data, and the fact that the resulting group was very small (only 14 women) and the pattern unusual. But a follow up was done, in a much large study, and the results, presented in the meeting, found no link to miscarriage in this group of women who had the vaccine two years in a row.
Other topics included extensive discussion of safety too. For example, the committee listened to data about a recently FDA approved six-in-one vaccine – Vaxelis – and heard safety data from six clinical studies and other studies. The committee asked about the causes of the six deaths in the trials and heard why they were determined unrelated.
One of the deaths was “unknown”, and the committee members asked what was the basis for determining it unrelated to the vaccines. The presenter explained that the death happened 44 days – that’s a month and a half – after the vaccine. The baby was found at home unbreathing. It was concluded that the cause was undeterminable, and the toxic report was negative. Based on timing, lack of evidence or plausibility of a link to the vaccine, it was determined unrelated.
Similar data were provided and discussed during the second day in the context of meningococcal vaccines. A discussion of the new Varicella zoster vaccine – Shingrix – included an even deeper delve into safety, with an examination of reports from the Vaccine Adverse Reporting System (VAERS) – a passive reporting system, and the Vaccine Safety Datalink (VSD) data.
VAERS reports were, for the large part (97.6%) for non-serious things. Among the serious reports, there was no pattern that would suggest the vaccine was causing serious harm. The rate of reporting was similar to that seen in other vaccines in that age group. All of this suggested there is no cause for concern about the Shingrix vaccine’s safety from these reports.
VSD data was examined using a Rapid Cycle Analysis. This is a monitoring tool, examining insurance codes to see if there’s anything that deviates from the norm in vaccine recipients. The goal is to catch signals – anything unusual after the vaccine.
Then professionals have to look at the signal and see if it shows a problem related to the vaccine or not. In this case, with one exception, there was no signal, including for most things the monitors examined specifically and actively – like strokes. But there was one signal – there were four cases of Guillain-Barre syndrome (GBS) after receiving the vaccine when 0.8 were expected according to the general rate in the population.
This was a lesson in what happens when there is such a signal. Investigators went and looked at each case. Out of the four, two were “historical” – they have had GBS in the past, long before getting the vaccine. One had symptoms indicating the GBS was in place before the vaccine. And one was in the risk window where it would be if the vaccine caused it. So did the vaccine cause that one case? It’s unclear at present. One case fits the 0.8 expected rates of GBS – so it may well just be what’s expected.
The conclusion was that the combination of the data suggested no concerning signals, but there is a need to monitor the GBS, to see if there’s a rare link. The one case, it was concluded, was not enough to change the existing policy – first, there was no clear evidence that the vaccine caused any GBS, especially since 0.8 cases are expected and since other sources of monitoring did not show a link, and second, the benefits are still higher even if there was a link. But there will be follow up.
So what’s the take home? Vaccine safety is taken seriously. Safety signals are followed carefully. The committee will not always conclude the vaccine caused harm, because they look at the data on that issue (or even if there is evidence of harm, the benefit may still be far greater), but they always look.
ACIP meeting observations – lengthy, deliberative process
Out of the many topics described, most were at a middle point in a long – potentially never-ending – process. Almost all referred to previous data provided in previous presentations.
For example, the Japanese encephalitis vaccine – the subject of three votes – was discussed in two previous meetings. The HPV vaccine has been discussed in many, and in this meeting, data was provided in preparation for the vote that may – or may not – take place in the summer. This is a careful, deliberative process of decision making. The meeting was almost completely about heavyweight substance (aside from an interlude in the end where the committee debated wording for one of the recommendations it voted on – deciding on wording is, of course, part of what they have to do, but it was not as complex or data heavy as the rest).
Acting under uncertainty
Very few decisions are ever made with complete information, and the vaccines decisions are no different. There are always areas of uncertainty and areas where the decisions need to be made on incomplete data.
It’s important to remember, here, that the decision not to decide – whether it’s not to decide to recommend a vaccine or not to decide to remove a recommendation of a vaccine – also carries risks. Not recommending a vaccine against a harmful infection means people will die or be hurt by the infection. Not removing a recommended vaccine whose harms are greater than benefits, or which is simply not cost effective, can lead to harms and costs.
The mandate of ACIP’s experts is to assess when there is enough information to make a decision, and, on the information provided – always incomplete, but at some point sufficient to make non-action problematic – to decide which option has the best combination of risks and benefits.
This meeting covered several topics where the uncertainty was especially glaring.
One of the first topics was allowing a new anthrax vaccine formula to be used in the emergency stockpile aimed for post-exposure prophylaxis. That is use after the fact in a case where a large number of people were exposed to anthrax – use in high-risk circumstances. The data on safety and effectiveness of the formula was limited, with more studies in the process.
The proposal was to have it available as back up to the existing supply because of the high risk if there is an exposure, but the vaccine would not be recommended for more general use until more data would be available. The point was raised that there is no safety data in children – and there is likely to be none since no study will be performed in children.
Given this lack of data, what should be done? The workgroup’s recommendation was that in spite of this uncertainty because the vaccine is to be used for existing exposure to a very, very dangerous infection, the benefits outweigh the risks. In other words, while there is a potential risk, not protecting children from anthrax when they were in an area where it was released to the air also has potential risks – and they are high.
In another session, the committee discussed whether the recommendation made in 2014 to give elderly adults PCV13 – a vaccine that covers 13 strains of pneumococcus bacteria – is still justified. This was an extremely complex and hard issue. I am not going to try and cover all of it. The heart of it is that there is clear evidence that PCV13 in children reduces invasive pneumococcal disease is both children and adults, and protects the elderly indirectly, through herd effects.
It’s less clear whether the recommendation that elderly adults be vaccinated has sufficient additional benefits to justify universal recommendation in that age group, though. On top of that are other uncertainties. Most cases now are caused by one strain, serotype 3 (this is a change from the pre-vaccine era), and the data for the vaccine’s effectiveness against this strain is mixed, making it unclear if the vaccine reduces the burden of disease in this serotype. Note that that does not mean the vaccine cannot prevent the disease in specific cases. Other studies suggested some effect on hospitalizations in the elderly. With the uncertainty about the effect on serotype 3, and with other questions about the impact of the vaccine when given directly to the elderly, there are a lot of questions about whether it makes sense to continue the routine recommendation.
To add to the complications surrounding this vaccine, there are new vaccines in late-stage in developments that may offer better protection to this age group, and a question that came up was whether dismantling the existing apparatus for giving the vaccines would make it hard to recreate a few years later, when the new vaccines are ready, and what the effect on compliance by program asked to create a program, dismantle it, and then recreate it would be. This was not voted on in this meeting.
The point of this discussion is to highlight that the experts on the committee often have to deal with situations of uncertainty, and it’s complex.
Anti-vaccine presence and comments
The meeting included a 75 minute public comment period and an opportunity to submit written comments via a government website. Since more people registered than the time available, a lottery was held to decide who would speak and in which order. Out of 20 speakers, 18 were anti-vaccine, and during those 75 minutes, they made comments criticizing the committee and claiming lack of vaccine safety. Orac covers some of the anti-vaccine comments.
While it was certainly an interesting experience to listen to their comments. However, in terms of contributing to the discussions of the committee, the comments were probably meaningless.
The vast majority of them did not even address the content of the committee’s discussions. Some of them addressed vaccines mandates, which are not within the committee’s purview, in language suggesting that they thought ACIP was doing something related to them. When they did address issues the committee discussed, their statements about them were extremely inaccurate and suggested a lack of understanding of the issues. When directed at a group of experts who have been following these issues for months or years, and understand the science, blatantly incorrect statements are unlikely to change minds.
Further, even comments that addressed the topics at hand did not make relevant suggestions. For example, although anti-vaccine activist Mr. Del Bigtree mentioned the Japanese encephalitis vaccine (though without seeming to understand that it was recommended only to specific categories of travelers, or why it was recommended), he made no suggestions about the three issues the committee was about to vote on in relation to that vaccine.
Looking at my notes, I can see no specific suggestion by any of the commenters that the committee could actually take on board. One person did argue against recommending Gardasil 9 for a later age, but her claims on that were incorrect – for example, she criticized the fact that the adjuvant is not licensed separately, though that’s the regulatory framework; and she said that the control for Gardasil 9 was aluminum. That’s incorrect – the control in the trials for Gardasil 9 was, correctly, Gardasil.
When replacing an existing vaccine with a new one, clinical trials look at non-inferiority – is the new vaccine as safe and effective as the old one? Giving the control group no protection from HPV infections, when there’s a licensed vaccine available, would have been unethical. The committee would know these claims are incorrect and ill-founded – as experts, they should know better than to be influenced by incorrect claims.
If the goal was to influence the committee, the comments were very poorly designed for it.
In reality, I doubt that the goal was to influence the committee. The comments were more likely an opportunity to express some of the commenters’ anger and to make YouTube videos to use among anti-vaccine activists. That is a little sad.
I expect quite a few of the people who attended are not wealthy, and traveling to ACIP and sitting through lengthy discussions their later comments suggested they did not understand very well for the chance to make a three-minute YouTube video of themselves giving an antivaccine speech – or hearing others give speeches – may well be putting strain on their resources and person. Seems to me, personally, like a poor investment of time and effort. Even if their intent was simply to intimidate and make committee members feel unsafe, I suspect the success was extremely limited. And such a goal would speak poorly about the movement they are part of, and the tools available to it.
The ACIP meeting was intense, informative, deliberative, and taught me that a lot of data and thought to go into each vaccine-related decision. It was reassuring and instructive. I think I learned a lot, and hope to attend future meetings (I cannot make the June meeting, but am already planning to make the October one).