In May 2018, I wrote an article about a Cochrane HPV vaccine systematic review that provided solid evidence that the human papillomavirus (HPV) vaccine was safe and effective. I considered the review to be one of the seminal pieces that support the use of the cancer-preventing vaccine. Moreover, most scientists in the biomedical field consider Cochrane systematic reviews (see Note 1) as near the pinnacle of the hierarchy of biomedical research.
Then, in early August 2018, several anti-vaccine, and more particularly vehement anti-HPV vaccine, “researchers” at Cochrane Nordic, a branch of the Cochrane Collaboration, went on the attack against the HPV vaccine. They published a paper in BMJ Evidence Based Medicine that blasted the Cochrane HPV vaccine systematic review.
I thought that this critique was without merit. Moreover, nothing they wrote diminishes the quality of the original Cochrane HPV vaccine systematic review. Once again, that systematic review provided us with solid, high-quality support of the fact that the vaccine is, indeed, safe and effective.
Even though the anti-HPV vaccine group provided some apparently cogent criticisms, it was clear that they had an agenda. Well, there has been more backlash against the anti-vaccine “researchers” in a long post by a scientist who studies and analyzes systematic reviews. And Cochrane itself responded to the criticism. Let’s take a look.
Cochrane HPV vaccine systematic review – the first defense
Hilda Bastian thoroughly examined the attack by Jørgensen et al. on the Cochrane HPV vaccine systematic review – she made seven key points about the criticism that I’ll summarize here.
- “The Cochrane review missed nearly half of the trials”. This was the first of the criticisms from Jørgensen et al. Bastian responded to this point by stating that the Cochrane Nordic group (see Note 2) did not provide any verifiable details of those “half the trials.” The Cochrane Nordic group thinks that the systematic review should have included research that isn’t published but taken directly from the clinical research data. Maybe they could have been included IF those “missing studies” fit into the inclusion criteria of this systematic review.
- “No included trial in the Cochrane review used a placebo comparator”. The “lack of placebo in the clinical trial” argument is a common trope of the anti-vaccine world. The Cochrane Nordic group also made a big stink about the controls in the included clinical trials, arguing that these controls contained adjuvants. Bastian replied with “Adjuvants are common in vaccines, including the HPV vaccine – and fears about them are a common target for criticizing vaccines, too. Arguing that these ingredients could be responsible for serious adverse effects is a staple of anti-vaccine fear-mongering, so there has been a lot of response on this.”
- “The included HPV vaccine trials used composite surrogate outcomes for cervical cancer”. The Cochrane Nordic group complained that the studies used “surrogates” for cervical cancer, called cervical intraepithelial neoplasms (CIN). This is another common anti-HPV vaccine argument that those lesions are not “cancer,” and they are often caught by a Pap test. Bastian responds with two important arguments. First, CIN diagnoses generally require 18 months of follow-up, tests, and biopsies – the significant psychological component of that time is completely ignored by the anti-vaccine crowd. And second, this surrogate is actually an excellent indicator of the potential for cervical cancer.
- “The Cochrane review incompletely assessed serious and systemic adverse events”. Bastian replied, “This one isn’t a reasonable complaint: assessment of anything is always going to be incomplete, and the Cochrane reviewers point out, rightly, that you can’t resolve uncommon safety issues based solely on trials. In fact, I don’t think the Copenhagen group identifies any serious flaw in this section of their critique.” I don’t think anything more has to be said.
- “The Cochrane review did not assess HPV vaccine-related safety signals”. There are so many large (maybe huge) epidemiological studies that found no safety signals, yet the Cochrane Nordic group continues to push that point. This group made a big stink about presenting “evidence” to the European Medicines Agency (EMA, Europe’s FDA) regarding (postural orthostatic tachycardia syndrome (POTS) and complex regional pain syndrome (CPRS) after HPV vaccinations. The EMA completely dismissed this nonsense, and no epidemiological studies have shown either POTS or CPRS to be more common after the HPV vaccination than in a non-vaccinated group.
- “Industry trial funding and other conflicts of interest”. I have always thought that rejecting clinical trials because of their funding source to be a form of bias, especially when those studies are examined carefully by researchers performing the HPV vaccine systematic review. Anti-vaccine “researchers” live in a world that doesn’t exist – someone has to pay for clinical trials, and it’s not going to be private donations.
- “Cochrane’s public relations of the review were uncritical”. This is kind of a weak critique, and Bastian quickly responds to it. “Cochrane did publish an editorial – which points to the need for other studies. Here’s their press release.” That’s a good scientific criticism.
Bastian ends with this summary/conclusion:
Where do I end up after all this? First, on the evidence: the HPV vaccine looks effective enough at preventing cervical lesions that it could reduce an enormous amount of suffering. And that’s wonderful.
Secondly, the critique: hmmm! When I first glanced at it, I was shocked and disappointed that Cochrane had seemed to mess up so badly on such an important review. But that flipped the more I dug into it, and it was the critique that increasingly shocked me.
Now that I’ve finished? I think it’s a hatchet job by people with several axes to grind, that needed better editorial peer review. I would be surprised if Cochrane’s investigation and update resulted in changed conclusions. But I look forward to reading it.
Cochrane HPV vaccine systematic review – the second defense
Let’s now turn to the best defense of the original HPV vaccine systematic review – and it comes from Cochrane itself (pdf). They undertook a month-long assessment of the original study in response to the Cochrane Nordic group’s commentary. I’m going to summarize their key points:
- The original HPV vaccine systematic review did not miss “nearly half of the eligible trials”. A small number of studies were missed due to the researcher’s focus on peer-reviewed studies published in scientific journals.
- The addition of data from outside those peer-reviewed studies does not make much difference to the results of the review for the main outcome.
- The trials comparators were unambiguously, transparently, and accurately described, as Bastian stated in her critique.
- The selection of outcomes for benefits was appropriate.
- The review included published and unpublished data on serious harms, and the findings on mortality post-vaccination were reported transparently and responsibly.
- The review was compliant with Cochrane’s policy on conflict of interest – there was none.
- Cochrane’s media coverage was cautious and balanced.
Cochrane stated that the group’s article substantially overstated its criticisms of the original Cochrane HPV vaccine systematic review. They concluded that:
In summary, we believe that the Cochrane Review represents a robust and accurate summary of the evidence.
Scientific debate is to be welcomed, and differences of opinion between different Cochrane ‘voices’ is not unexpected. However, public confidence may be undermined, unnecessary anxiety caused, and public health put at risk, if that debate is not undertaken in an appropriate way. This is especially true when such debates take place in public. There is already a formidable and growing anti-vaccination lobby. If the result of this controversy is reduced uptake of the vaccine among young women, this has the potential to lead to women suffering and dying unnecessarily from cervical cancer.
Summary
The Cochrane Nordic group’s critique of the HPV vaccine systematic review was without merit. I thought it was another attack by that group on the safety and effectiveness of the cancer-preventing HPV vaccine.
The responses by Hilda Bastian and Cochrane Reviews were strong and to the point – there was simply nothing there.
Of course, this won’t stop the anti-vaccine religion from using that poorly analyzed critique as definitive evidence that somehow the HPV vaccine is useless. That’s dangerous to men and women who can be protected from deadly cancers.
Can we now move away from this ridiculous anti-HPV “research?”
Notes
- The Cochrane Reviews is considered one of the premier organizations that perform systematic and meta-reviews in the biomedical sciences. If I am looking to determine if there is evidence supporting a medical claim, I look here first. Now, I’m a scientist, so I don’t take their conclusions at face value – they have made egregious errors in systematic reviews of acupuncture quackery in the past. But like all scientific literature, one must examine a systematic review (whether published in Cochrane or any other journal) with a critical eye. Is there bias in including or excluding data? Do they overstate the conclusion? Do they rely upon unusual or bad statistical analyses?
- I’m going to refer to the group who wrote the critique of the HPV vaccine systematic review as the “Cochrane Nordic group.” Officially, although all three authors are employed by Cochrane, their anti-vaccine opinions do not represent Cochrane or Cochrane Nordic.
Citations
- Arbyn M, Xu L, Simoens C, Martin-Hirsch PP. Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors. Cochrane Database Syst Rev. 2018 May 9;5:CD009069. doi: 10.1002/14651858.CD009069.pub3. [Epub ahead of print] Review. PubMed PMID: 29740819.
- Jørgensen L, Gøtzsche PC, Jefferson T. The Cochrane HPV vaccine review was incomplete and ignored important evidence of bias. BMJ Evid Based Med. 2018 Jul 27. pii: bmjebm-2018-111012. doi: 10.1136/bmjebm-2018-111012. [Epub ahead of print] PubMed PMID: 30054374.
