Last updated on November 27th, 2017 at 12:17 pm
Late last year, I spent a couple of thousand words critiquing an awful anti-vaccine paper that attempted to claim that vaccinated kids were sicker than unvaccinated kids. Soon after the paper was published, a huge kerfuffle arose on Twitter, which pointed out the false conclusions, bad experimental design, and high levels of bias made by the authors. Within days of publishing the article, the journal retracted it (see Note 1).
Tell me if you’ve heard this before.
Not two days ago, I spent a couple of thousand words taking down an anti-vaccine paper that tried to claim that vaccinated kids were sicker than unvaccinated kids. Soon after the paper was published, the journal retracted it (see Note 2).
Oh wait, you had heard it before. The papers were the same one, where the primary author went shopping for journal after journal to get it published. Why so persistent? The primary author, Anthony R Mawson, from Jackson State University, has a long, established background of supporting the Andrew Wakefield along with anti-vaccine research. In 2011, Mawson had actually filed a lawsuit against the Mississippi State Department of Health in which he claims that there is a need for more studies on vaccines (including his own design, apparently), and that the Department of Health interfered with his getting a contract renewal at the University of Mississippi Medical center. The suit was dismissed in 2012.
Mawson actually has a Master’s and Doctorate in Public Health, specializing in epidemiology. You’d think he’d know better about vaccines, but I guess being a sycophant of Mr. Andrew Wakefield makes him deny his scientific background. Then, he goes out and invents a bad experimental design just to support his nonsense beliefs. And to be more endearing to Wakefield I suppose.
Now, you, my reader, are probably thinking, “OK, got it, let’s talk about GMOs.” Except, I have to drag you back in, and remind you that myths, tropes and memes in the anti-vaccine world are like zombies. They never quite die, biting one anti-vaccine cheerleader after another. Even a dead paper, like Mawson’s, will arise again somewhere.
As the curmudgeonly Orac just wrote,
One of the most frequent topics for posts like this is a pseudoscientific or just plain bad study that, despite being retracted, keeps rising from the grave, like the proverbial zombie. I call them, appropriately enough, zombie studies. Depending on my mood when I write posts like this, I often add imagery featuring zombies (or, if you’re into The Walking Dead, walkers).
I can’t resist a good Walking Dead metaphor.
Let’s look at this zombie/walker anti-vaccine paper. Because you just know, like the end of the zombie movie, the story drops hints for Part II. Or Part II. Or Part XXI.
How did an anti-vaccine paper get retracted twice?
The simple answer is that it was published twice in predatory journals, whose standard of publication is not much more than if the author can pay the fees to publish their paper. Mawson’s article was published for the first time in Frontiers in Public Health, a part of the Frontiers Media empire. It was published for the second time in the Journal of Translational Science, a part of OAT. Both Frontiers Media and OAT are predatory publishers according to the definitive Beall’s List of predatory publishers. A predatory publisher is an “exploitative open-access publishing business model that involves charging publication fees to authors without providing the editorial and publishing services associated with legitimate journals.”
In case you missed my articles about this before, let’s look at Frontiers Media and OAT:
- Frontiers in Public Health (FPH) is not indexed in PubMed. The Journal of Translational Science (JTS) is not indexed in PubMed. PubMed is one of the most valued resources in searching for biomedical articles, even in obscure journals. Without PubMed, it is extraordinarily difficult to find the article. Of course, there’s no reason given for it’s exclusion from PubMed, but we can speculate that possibly the science is bad.
- Both FPH and JTS lack an impact factor. The impact factor gives us a relative understanding of the quality of a journal – generally, the impact factor tells us approximately how many times an average article published in a journal is cited per year. A journal like Nature has an impact factor of 36, which means the average article is cited 36 times a year. Why is this important? We know that more pivotal and valuable research is cited many times and often repeated by other scientists, especially in new publications. This is how science develops solid evidence, by examining and repeating the findings of others.
- As we mentioned above, both FPH and JTS are published by predatory publishers (one of the possible reasons that the journals are not indexed).
- Low impact factor, predatory journals are generally dismissed by scientific researchers because the peer-review is weak. These journals are often abused by researchers whose data has been rejected by one or more of the respected journals, so they settle for these low quality versions. In the academic community, publications such as these often cannot be used as evidence of qualification for tenure.
Clearly, neither FPH or JTS were actually interested in what Mawson had written. If they were, either or both of them would have rejected the articles in the first place. But here we are, both articles were “published” (or whatever verbiage they use), and both were retracted (or scrubbed from the internet).
Remember, this anti-vaccine paper was garbage
Just in case you’re anti-vaccine, and you think that there’s some conspiracy against “evidence” that vaccines are dangerous, get over yourself. These two journals are awful, and it’s entertaining that Mawson’s article was so bad that it fell beneath the journal’s miserable standards.
Let me remind you of how terrible this article was (before being retracted twice).
- This study intended to “compare vaccinated and unvaccinated children on a broad range of health outcomes, and to determine whether an association found between vaccination and neurodevelopmental disorders (NDD), if any, remains significant after adjustment for other measured factors.” This is a good start, and if done well, it could provide us with more information about any potential links between vaccines and NDDs such as autism spectrum disorder. But this is not a well-designed study to answer that question.
- The study surveyed 415 mothers of 666 children educated at home. And the paper goes downhill quickly. First of all, we are relying on data from a group, which may or may not be biased versus a random sampling, that home schools children. There are lot of reasons why parents do this, but are they representative of the population at large? Second, it relies upon the memories and, frankly, biases of the mother being surveyed – it doesn’t rely upon data in actual medical files of each of these children, which can be considered to be somewhat “scientific.”
- Parents who do vaccinate may be less reluctant to visit their physician to get diagnosed for diseases, while a non-vaccinating parent may choose to ignore any health issues as minor. Thus, there may be an unintentional bias towards health care issues for vaccinated children.
- The groups (vaccinated vs. unvaccinated) are not randomized which will lead to observational and confirmation bias.
- The population for the study is quite small to tease out data on very rare conditions. A meta-review that included nearly 1.3 million children, using real medical records to establish both vaccination status and ASD status, found that there was no difference between vaccinated and unvaccinated groups with respect to incidence of autism.
- Questionnaires are not the best way to gather data about medical issues – in fact, the use of questionnaires for epidemiological studies is frowned upon by many researchers. The reasons for this are many, but they include a reliance upon the memory of the participant for events that may be not be recalled correctly. In addition, there might be intentional or unintentional bias of the survey participants towards relating a condition to vaccines or ignoring one in unvaccinated kids. Better done, and frankly, more expensive, epidemiological studies rely upon medical records to reduce errors in memory and bias. This is a critical point – the best studies that show no link between vaccines and autism rely upon actual medical data for each child, not on a questionnaire which may or may not be tracking accurate information.
- At best, this is observational data, which are not the type of data used to reject or accept an hypothesis. The CDC uses surveys to determine vaccination uptake, for example, but they used a non-biased methodology to gather data. They surveyed nearly 15,000 children, and were given permission to contact the health care provider to get more information.
- Mawson’s study found that 39% of children surveyed were not vaccinated. Now if we accepted this data without analysis, we’d think that the vaccination rate has dropped. However, that’s not what we’ve seen in other, better-designed surveys, such as the CDC’s massive ones, which showed that vaccine uptake was between 85 and 95% depending on the vaccine and the area. With such a difference between this study and better designed ones, one would have to believe that the study with a much smaller number and an apparently biased population would be the outlier.
- There was a lack of analysis of potential cofounders which might bias the results. Were there other differences between he vaccinated and non-vaccinated groups that could have biased the results one way or another?
- The use of odds ratio by the researchers is curious and may have biased the results. The only reason to use OR is to inflate the difference between both groups. Real epidemiologists prefer relative risk (RR) which provides us with a much more sensitive understanding of the incidence of a condition in a vaccinated group vs. an unvaccinated group. For example, the RR for ASD from this study (if the data actually were useful, which it isn’t) is 3.1, whereas the OR is 4.2.
- The over reliance on p-values is always a red flag to researchers. With so many p-values, the probability of false positives is fairly large. Because the population for the study was so small, it’s also worth nothing that the OR had relatively large error brackets, most of which crossed near unity, that is, 1.0, indicating no difference between the groups.
- The authors seem to overlook or ignore the point that the vaccinated children had much lower risk of diseases like chickenpox and measles, which we know bring with them significant risks of long-term health issues. Even if we accept this poorly designed study as providing us with evidence of harm from vaccines, one would have to balance it against the benefits of avoiding vaccine preventable diseases. Of course, this study is so bad, it’s hard to accept anything useful from it.
There’s more, I’m sure. I just get tired of tearing apart these bogus articles, when it’s really hard to get over the fact that it’s been retracted twice by awful predatory publishers. Seriously, that should be the end of the story.
Summary
As the crotchety Orac wrote recently about Mawson’s article,
I’ll conclude by pointing out yet again that it is a myth that there are no studies comparing the health of vaccinated children compared to unvaccinated children. In fact, there have been several. It turns out that they don’t show what antivaxers think a vaxed/unvaxed study will show. Basically, all of the vaxed/unvaxed studies not done by antivaccine-friendly scientists or quacks have shown either no differences in the prevalence of neurodevelopmental or chronic diseases between vaccinated children and unvaccinated children or have actually found better health outcomes in the vaccinated population. Mawson concludes by arguing that further “research involving larger, independent samples and stronger research designs is needed to verify and understand these unexpected findings in order to optimize the impact of vaccines on children’s health.” Mawson’s study is so biased, flawed, and incompetently carried out and analyzed that its results can be discounted as almost certainly worthless. It doesn’t provide the rationale for “more studies.” Quite the contrary.
Yet, that’s how antivaxers are spinning it, as they always do.
Same as it ever was.
There really are plenty of good articles, published in real, indexed, high impact factor journals that show that there is little or no difference between the health of vaccinated versus unvaccinated children. We should also include that nagging point that vaccinated children don’t generally catch deadly, disabling vaccine preventable diseases.
We also have boatloads of evidence, again, published in major journals, that dismiss any link between vaccines and autism, which feels like the basis of Mawson’s junk article. So, if you want to use that rubbish anti-vaccine article as the basis of your “evidence” that vaccines are dangerous – well, you really need something more convincing. Mawson’s article was retracted twice by the detritus of predatory publishers, so it would be laughable to use it as real evidence.
But we all know that this zombie article will rise again. They eat brains, and apparently they’re looking for nutritious anti-vaxxer brains. But zombies aren’t real, except on TV and movies. So the anti-vaccine world keeps pushing zombie tales – because they think that by Part XXI, we will be bored with the zombie myth and not try to take it down. Don’t count on it.
Notes
- Actually Frontiers Media noted that the study was only “provisionally accepted but not published.” That might be what they think, but the article appeared on their website, then disappeared.
- Technically, we can’t tell that the article was retracted. It’s just been scrubbed from JTS’ website, as if it never existed. But the internet never forgets.
