This article, about the anti-vaccine book, “Turtles All the Way Down,” was written by Dorit Rubinstein Reiss, Professor of Law at the University of California Hastings College of the Law (San Francisco, CA), who is a frequent contributor to this and many other blogs, providing in-depth, and intellectually stimulating, articles about vaccines, medical issues, social policy, and the law.
Professor Reiss writes extensively in law journals about vaccination’s social and legal policies. Additionally, Reiss is also a member of the Parent Advisory Board of Voices for Vaccines, a parent-led organization that supports and advocates for on-time vaccination and the reduction of vaccine-preventable diseases. She is also a member of the Vaccines Working Group on Ethics and Policy.
This post opens by providing some context for the anti-vaccine book, “Turtles All the Way Down,” and then provides a thorough review of its first chapter. One thing confusing about the book is that the references are provided in a separate document, attached here. So you need to go and look them up (as I did) to be able to fact-check.
Background
Several years ago, Professor Yehuda Shoenfeld, an immunologist that has become friendly to the anti-vaccine movement, caused a scandal in Israel by publishing in a respected medical journal he was editing a review by two criminologists – without subject-matter expertise – an anonymously-authored anti-vaccine book called “Turtles All the Way Down.”
A few years later, the book was translated into English, with a foreword by two well-established anti-vaccine activists, Mary Holland, who has been active in anti-vaccine circles for a long time and is now President and General Counsel of the anti-vaccine organization Children’s Health Defense, and Zoey O’Toole, who has been active in anti-vaccine blogging.
Page numbers in this post refer to the first chapter in this English version, linked here. Even though it has been around for a while, I did not write about this book before, because it is making (problematic) claims about science, and I am a legal academic.
Anti-vaccine “Turtles All the Way Down”
But since “Turtles All the Way Down” is being floated in the anti-vaccine sphere again, I thought I would address it. I read the first chapter of the book. In essence, it repeats what has become a common anti-vaccine talking point, the claim that vaccines are not properly tested because they are not compared to true placebos in randomized clinical trials. Then, the chapter argues that this is done intentionally to hide vaccine risks.
This claim has been addressed before, and in-depth, by Skeptical Raptor, Orac, Dr. David Gorski, Kathy Hennessy, Vaxopedia (and also here), and the Vaccine Education Center in the Children’s Hospital in Philadelphia talks of it in its discussion of vaccine development. So this is far from a new talking point in the anti-vaccine lore.
But since the book repeats the claim in a way that may well mislead readers who have not had a chance to follow the discussion, it’s worth delving into. In essence, the claim, as presented by the book, is misleading in three ways.
First, the book’s view of a “true placebo” is wrong and misleading – contrary to the book’s claims, a clinical trial can be valid even if it does not use a saline solution as a placebo as a comparator, and in fact, sometimes using such a comparator is unethical.
Second, the suggestion that randomized clinical trials are the only way to obtain data on vaccine safety is mistaken. When the book addresses very old vaccines, created before the routine use of randomized clinical trials, the book is wrong to suggest that decades of data collected on these vaccines are not good evidence about them, evidence that would make conducting a clinical trial depriving children of these vaccines unethical.
Third, the book has many inaccuracies – probably unintentionally; not knowing who the authors are, there is no basis to assume expertise. To give one example I will go back to when writing about polio vaccines the authors completely ignore the large placebo-based trials conducted for the Salk vaccine – whose formula is still the one used in the United States – in the 1950s. Instead of separating that section, I will build some of those errors into the discussion – for example, pointing out that the book missed that the first Hib vaccine the authors discussed is a newcomer, a 2009 vaccine, and should have been compared to the earlier existing vaccines from the 1990s – that’s not an error.
In addition to these three glaring errors, the book makes two deeply flawed assumptions. First, much of the discussion – emphasizing the risks of vaccines as the only consideration in clinical trials- rests on the premise that vaccines have risks but not benefits. In other words, the book assumes that there is no cost to not vaccinating children against disease. In reality, since vaccines protect children against dangerous diseases, not vaccinating a child means leaving a child unprotected. That is a cost. In suggesting that children not be vaccinated against, for example, tetanus, diphtheria, and pertussis, the book is assuming that leaving children at risk of, say, tetanus – with 10% mortality even with the best treatment – is ethical. That’s problematic.
Second, the book assumes that vaccines are all of a kind. For example, one of the arguments against having a trial in which children are given other vaccines as a control, or other vaccines – not as a control – as part of the routine schedule, the book argues that this is equivalent to testing cigarette safety by comparing one brand of cigarettes to another.
But that comparison is simply false. Vaccines are not all the same. Each vaccine was developed to address a different risk, and each has its own features. Arguing that giving children diphtheria, pertussis, and tetanus vaccines as part of the routine schedule undermines comparing children who got the Hib vaccine to children who did not (English version, p. 59) is simply incorrect; the DTP vaccine (for diphtheria, tetanus, and pertussis) has, at the point of the Hib trials (the ActHIB vaccine, one referred to here, was licensed in 1993; PedVaxHib, the other, in 1998) the DTP vaccine was in use for over 40 years.
The argument that this very different vaccine, in use for decades and with abundant data about it, should not be given to children in the Hib trial is problematic in several ways. First, it would be unethical not to protect children from diphtheria, tetanus, and pertussis just because they’re part of a Hib vaccine trial. Second, since all children are getting the DTP vaccine, and since it’s a very different vaccine whose safety profile is known, it does not undermine or interfere with the comparison between children who did or did not get the Hib vaccine. This is a very valid comparison.
Finally, potentially because of a lack of knowledge, the authors treat the numbers of reported adverse events as showing what the vaccine did or did not cause. For example, in discussing rotavirus vaccine trials on p. 68, the authors criticize the trials for using the vaccine’s solution without the active ingredient as the control. Then they say that “about 1 in 30 control group subjects experienced a “severe” medical event (a rate which was even slightly higher than that of the trial group), and a similar proportion of participants was hospitalized.”
Then on p. 70, they refer to this saying “These compounds had significant potential to cause harm, as demonstrated in the trials. (Remember, 1 in every 30 or 40 control group subjects experienced a serious adverse event).” This assumes that serious adverse events would be related to the substance given – but that’s not true.
As explained by the authors of the study discussed, “Serious adverse events were defined as any new health-related problems that resulted in death, were life-threatening, necessitated hospitalization or prolongation of existing hospitalization, or resulted in disability or incapacity.” In a trial with tens of thousands of infants, some new problems are expected, and in reality, at least some of the problems investigated – those related to diarrhea – are problems in which the presence or absence of the active vaccine component – a weakened live rotavirus virus, a virus that causes those issues – was of prime importance. The book authors also assume that the deaths could be considered related to the substance; but in one example discussed in the article, 22 of the deaths in both vaccine and placebo groups were from pneumonia – something that cannot be reasonably related to the vaccine or the buffer solution.
The anti-vaccine “Turtles All the Way Down” is wrong on what is an appropriate control
The anti-vaccine book, “Turtles All the Way Down,” makes several claims on what is an appropriate control, but I will focus on just two, to avoid writing another book.
First, the book argues that the only valid control is saline (or sucrose) control. Second, while the book acknowledges that experts point out that when there is an approved vaccine it’s unethical to deprive a control group in the trial of it, the book then – based on an apparent misunderstanding – argues you should have a placebo group anyway. Both claims are wrong.
Valid Controls:
The book argues that “…an RCT (randomized clinical trial) in which the control group receives an inert placebo is designed to answer the critical question of How many adverse events does the new vaccine cause? Of course, we should keep in mind that trial results are no more than a good estimation.” (p. 44) It goes on to argue that “[a] new pediatric vaccine is never tested during its formal approval process against a neutral solution (placebo).
Comparing a trial group to a control group that was given a compound that is likely to cause a similar rate of adverse events facilitates the formation of a false safety profile.” As will be discussed below, that’s not actually true. Even above, I gave the example of a trial that compared children who got the Hib vaccine and children who did not. But more fundamentally, it’s untrue that you have to use an inert placebo.
As the Skeptical Raptor explained elsewhere:
The goal of a control group is twofold. First, to examine if the vaccine is effective in preventing disease. Here, comparing the vaccine to something that wouldn’t prevent that disease is what is needed. Second, to examine if the vaccine is safe. Here, comparing the vaccine to something known to be safe is needed.
There are two potential alternatives used. One is a solution that only contains the additive to the vaccines compared to a solution which includes also the active ingredient. Why is a solution with the additional ingredients a valid control? Because generally, the tiny amounts of additional ingredients in vaccines are a known quantity. Aluminum adjuvants, for example, have been used since the 1930s and their effects aren’t a mystery. The question is what does the active ingredient in the vaccine do.
The book addresses three types of what they see as non-saline placebos. The use of another vaccine, and the use of the solution of the vaccine without the active ingredient, and argues that both distort the results. It’s wrong on both counts. The book also argues that giving another vaccine to the children in the trial, even when some children are not given the trial vaccine, is not a valid test, and that, too, is wrong.
In 2013, the World Health Organization convened an expert panel to inform it on the ethics of placebo use in vaccine trials. As a starting point, and as discussed below, the report made it clear that it is not always ethical to give a placebo – and in fact, it’s unethical where there’s an existing safe and effective vaccine. Among other things, the panel addressed the use of a control vaccine, in Box 3, and said:
In place of a placebo, a vaccine against a disease that is not the focus of the trial is given to participants who do not receive the trial vaccine. Typically the control vaccine is a licensed vaccine for which efficacy has been demonstrated and the safety profile is well characterized. The motivation for using active rather than inert “placebos” is to fulfil the ethical duty of beneficence and, sometimes, to avoid giving an injection with an inert substance.
A methodological disadvantage, however, is that trials using these types of placebos provide a less perfect control. It may be difficult or impossible to assess fully the safety and reactogenicity of the trial vaccine, although its efficacy can usually be assessed satisfactorily. Such trials may also be less acceptable to regulators. Some regulators and/or public health authorities may prefer data from a placebo-controlled trial on which to make decisions whether or not to approve or adopt a vaccine.
In other words, the use of an alternative vaccine is a recognized control approach, with advantages and disadvantages. The advantage is that the control group gets a benefit.
The concern is that the results in terms of safety may be harder to assess – but that concern is very dependent on what the comparator vaccine is. For example, the book (p. 62) criticizes a hepatitis A vaccine trial for using a hepatitis B vaccine as a control. But at the time of the trial (1995) the company’s hepatitis B vaccine had been licensed for six years, since 1986, after its own trials; that’s a well-known quantity and a very, very different vaccine — the hepatitis A vaccine is a whole inactivated virus vaccine, while hepatitis B vaccine has a single protein from the virus.
Nothing is misleading about comparing a well-known vaccine (which has, as its main side effect, the potential to very rarely cause a severe allergic reaction) with a new vaccine: the safety results would be valid. They’re different enough – and the hepatitis B vaccine was well known enough – to make the control valid.
There is also no problem in using a known solution to compare the vaccine too. Generally speaking, the solution the vaccine is in would not contain anything problematic. Let’s use the rotavirus vaccine trials, where the authors criticize the trial conductors for using, as a control, the vaccine solution without the active ingredient (pp. 68-70).
Here are the ingredients of that solution for Rotarix, one of these vaccines: “ROTARIX also contains sucrose, di-sodium adipate, Dulbecco’s Modified Eagle Medium (DMEM), and sterile water. “
None of these is an issue. Sucrose and sterile water are not an issue under the authors’ own description. Neither are the other ingredients. DMEM is not hazardous. Neither is di-sodium adipate. The entire vaccine is 1.5 milliliters – including all the ingredients, including the sterile water. For comparison, a teaspoon is 5 milliliters; so we are talking about a solution that is less than a third of a teaspoon of non-dangerous ingredients. It’s unclear why the authors think it’s dangerous enough to be an issue or not a valid control.
One common criticism is the use of aluminum-containing solutions as a control. For example, HPV vaccines were tested that way. First, there has also been at least one trial that compared the HPV vaccine to a saline placebo. But there are valid scientific reasons to use a well-known and safe ingredient as a comparator.
Finally, the most common criticism in the book was that children getting the trial vaccine were generally also given their routine childhood vaccine, which the authors think diluted the results. This criticism was made, for example, about Hib vaccines (58-59), polio vaccines (p. 60), pneumococcal vaccines (p. 61), and hepatitis A (pp. 62-63).
This criticism is wrong in several ways. First, as explained above, participation in a vaccine trial for a given vaccine is not a justification for depriving children of protection from other diseases, and the only reason to argue otherwise is if you completely discount the benefit of protecting children from disease – assume away all vaccine benefits.
Second, not only would it be unfair to deprive children in a trial for a new vaccine from protection from diseases with approved vaccines, we want to know the safety of a given vaccine on the background of other vaccines – since children will get those, too. In other words, we want testing of a new vaccine with the schedule – this is called concomitant studies, and manufacturers are required to do them (though anti-vaccine activists deny it in another talking point, where they claim vaccines are not tested together).
New versions or combinations of existing vaccines should be tested against the existing vaccine.
Ethics of vaccine clinical trials
The book argues that vaccines to replace existing vaccines should be subject to a trial with three arms, one of them placebo, because, argue the authors, a trial against an existing vaccine cannot give you the real rate of adverse events. The book says: (p. 45)
In order to determine the true rate of adverse events of a new generation vaccine, a three-arm trial must be conducted, combining the two methods described above. In this kind of trial, subjects would be randomly allocated into three groups, one trial and two controls: The trial group would receive the new generation vaccine, the first control group would receive the current vaccine, and the second control group would receive an inert placebo. This trial design is considered to be of excellent quality, as it measures both the absolute rate of adverse events (comparing the new vaccine to the placebo) and the relative rate (comparing the new vaccine to the current vaccine).
This is incorrect. First, here is the World Health Organization on this. In 2014, the World Health Organization said:
Placebo use in vaccine trials is clearly unacceptable when (a) a highly efficacious and safe vaccine exists and is currently accessible in the public health system of the country in which the trial is planned and (b) the risks to participants of delaying or foregoing the available vaccine cannot be adequately minimized or mitigated (e.g. by providing counselling and education on behavioural disease prevention strategies, or ensuring adequate treatment for the condition under study to prevent serious harm). In this situation, a placebo-controlled trial would not address a question that is relevant in the local context, namely how the new vaccine compares to the one that is currently in use, and participants would be exposed to unacceptable levels of risk from delaying or foregoing a safe and effective vaccine that is accessible through the public health system.
This is exactly the situation where there is either an existing old vaccine or when manufacturers want to replace single vaccines with a combination of vaccines. In those situations, there is an available prevention against the disease in question. This is not unusual to vaccines: as the World Health Organization said in 2022,
The benefits, risks, burdens and effectiveness of a new intervention must be tested against those of the best proven intervention(s), except in the following circumstances:
–
Where no proven intervention exists, the use of placebo, or no intervention, is acceptable.
–
Where, for compelling and scientifically sound methodological reasons, the use of any intervention less effective than the best proven one, the use of placebo, or no intervention is necessary to determine the efficacy or safety of an intervention, and the patients who receive any intervention less effective than the best proven one, placebo or no intervention will not be subject to additional risks of serious or irreversible harm as a result of not receiving the best proven intervention. Extreme care must be taken to avoid abuse of this option.
The WHO is quoting the Declaration of Helsinki here, a leading document on ethical medical experimentation on human subjects. In other words, under international norms, what the authors are suggesting – which is, in essence, depriving children of the trial of existing protection – is unethical.
When there is an existing treatment, or in this case, a preventive, it must be given. Not protecting young children from dangerous diseases when there is an approved vaccine available is unethical. As a reminder, we create vaccines for diseases that kill and harm children. Not providing them protection is only convincing or plausible if you assume away the risks of not vaccinating, if you assume away the risks of tetanus, diphtheria, polio, Hib, and the other diseases – and that is exactly what the authors are doing here.
Where did the authors, therefore, get the misguided idea that experts recommend a three-arm trial? Their reference collections link to this document, which is titled “International Conference On Harmonisation Of Technical Requirements For Registration Of Pharmaceuticals For Human Use” and is from 2000.
As a first point, this document is not about vaccine trials and is older than either WHO guidance refers to above. As a second point, it does not say what the authors of the book seem to think, which is that this is the recommended trial design when comparing an existing vaccine to a new one. Three-arm trials are first mentioned on p. 12 of the document, where the document says: “A useful approach to the assessment of assay sensitivity in active control trials and in placebo-controlled trials is the three-arm trial, including both placebo and a known active treatment, a trial design with several advantages.”
This is raised in the context of identifying the effectiveness of a new treatment, not safety. The heading there is assay sensitivity, defined on p. 7 as “Assay sensitivity is a property of a clinical trial defined as the ability to distinguish an effective treatment from a less effective or ineffective treatment.”
In other words, the document does not suggest this design is of “excellent quality” for identifying the rate of adverse events. On p. 15, the document continues:
As noted in section 1.5.1, three-arm trials including an active control as well as a placebo-control group can readily assess whether a failure to distinguish test treatment from placebo implies ineffectiveness of the test treatment or is simply the result of a trial that lacked the ability to identify an active drug. The comparison of placebo to standard drug in such a trial provides internal evidence of assay sensitivity. It is possible to make the active groups larger than the placebo group to improve the precision of the active drug comparison, if this is considered important. This may also make the trial more acceptable to patients and investigators, as there is less chance of being randomized to placebo.
Again, this does not suggest this design is good for identifying the true rate of adverse events or safety but focuses on effectiveness. And it does not say anything about vaccine trials or comparing an old to a new vaccine.
These are the only places in the document where a three-arm trial is suggested. So, very far from recommending a three-arm trial for existing vaccines to assess safety, the document mentions this design in passing as a way to solve problems in identifying the effectiveness of existing treatments and does not go so far as to recommend it generally. The document is also substantially older than the discussions of vaccine trials design by the WHO referred to above.
This error – wrongly suggesting it’s appropriate to give a placebo in the context of an existing vaccine – addresses many of the claims the book makes about vaccine testing. The book criticizes testing of DTaP compared to DTP (p. 56) – but that was exactly the right comparison, old vaccine to new.
On this, the book also criticizes not having randomized clinical trials for the older DTP vaccine – but that is simply because the vaccine is very old, and at the time such trials were not run. After decades of data on the vaccine, it would be unethical to stop giving it to perform such trials – because it is not unknown. There is strong evidence that DTP protects children from diphtheria, pertussis, and tetanus, and is reasonably safe.
The book similarly criticizes the Hib vaccine on similar grounds, starting with GSK’s Hib vaccine, Hiberix, which it criticizes for being compared to other Hib vaccines. But Hiberix was licensed in 2009. As mentioned above, the other Hib vaccines were licensed in 1993 and 1998. Of course, the later, new vaccine was compared to the existing ones — that was correct. The only criticism of those vaccines the book has is that the trials gave the children other vaccines – DTP in one case, DTP and OPV (oral polio vaccine) in another – and as explained above, that’s not a problem. The children were not supposed to be deprived of routine vaccines because of participating in a trial.
For the polio vaccine, the book says “IPOL vaccine, by Sanofi Pasteur, is the only brand currently licensed and used in the United States. Its package insert does not mention any pre-licensure RCTs that were performed for the vaccine. Wasn’t the vaccine clinically tested before it was introduced into the US schedule in the early 1990s? Well, no. According to a document released by the FDA in 2018 following a Freedom of Information Act (FOIA) request, the vaccine underwent two clinical trials.”
But the inactivated polio vaccine was not introduced to the United States in early 1990. It was given in the 1950s until it was replaced by the oral polio vaccine, and before it was given, it was subjected to very large clinical trials. Many of them used a placebo-controlled design. Then, in the 1960s, it was replaced by Dr. Sabin’s oral polio vaccine – which was also subjected to large clinical trials. I’m not sure what kind of trials, so I am not sure these were placebo-controlled, but these were not without trials.
The insert of IPOL does mention multiple clinical trials, too, on p. 5-6, but they are not compared to a placebo – one set of trials compared the vaccine, correctly, to OPV, and the other looked just at effectiveness. But again, this is not a new vaccine. This is a vaccine to replace an existing vaccine that replaced another existing vaccine – and the original vaccine had large clinical trials.
For the vaccine against pneumococcal disease, the authors criticize the testing of the existing vaccine for comparing it to the old vaccine – but that is, again, the right comparison. They also correctly point out that Prevnar was tested in large trials comparing those who received it to those who received another vaccine, an investigational meningococcal vaccine. The authors criticize this, and this is a case where I would like to have more data about the choice, since I, too, am wondering why choose another investigational vaccine as a comparison. But there are studies comparing children who received Prevnar 7 to children who did not.
Hepatitis B vaccine — as with Hib vaccines and polio vaccines, the book discussion here does not start with the first vaccine. The first hepatitis B vaccine was a Merck vaccine – a plasma vaccine – in 1981. This plasma vaccine was tested against a placebo. The second hepatitis B vaccine, by Merck, was licensed in 1986. This vaccine was licensed in 1989.
Since it’s a vaccine to compete with an already existing, licensed vaccine, the correct comparison is to the existing vaccines. So criticizing it for doing just that, as the book does, is an error. Similarly, the next vaccine mentioned is GSK’s combination vaccine for hepatitis A and B – which should have been compared to the individual vaccines and was.
The only last thing I will mention is that the book’s criticisms of MMR vaccine trials were already addressed by my friend and colleague, Dr. Vince Iannelli. Dr. Iannelli found one placebo trial of MMR, but pointed out, correctly, that the larger trials were of the component vaccines, and the trial for MMR compared the component vaccines to the combination – as is appropriate: when you combine vaccines, the right comparison is whether the combination vaccines are as safe and effective as the individual vaccines.
You do not get to deprive children of protection from disease, as Orac explained. Similarly, in the section on MMR (pp. 63-64), the book criticizes comparing the four-in-one vaccine to MMR and chicken pox vaccines individually – but that’s the right comparison.
The Book is Wrong to Suggest Only RCTs Matter
The book would like to dismiss evidence on vaccine safety besides randomized clinical trials. But that, too, is wrong. This approach has been referred to by Orac as “methodolatry”, which he defines as “the ‘profane worship of the randomized controlled clinical trial (RCT) as the only valid means of investigation.’” The book uses this to dismiss, for example, the safety evidence about DPT, a vaccine that was brought into use before randomized clinical trials became routine. But this is unconvincing, for several reasons.
- Clinical trials are not enough to identify rare side effects. That is why every vaccine, with no exception, is subject to phase IV studies and monitoring after licensure. Clinical trials cover, at most, tens of thousands of people – and those are large trials. But they will not detect side effects in 1:100,000 or less. If we were to limit our attention to data to clinical trials, we would have less safety examination and less safety data, not more. We want to go beyond the clinical trials.
- Sometimes, issues are not known or do not receive attention when the clinical trials are done – either because the issue is not on the horizon or because the vaccine was later licensed for other purposes. For example, when many of the childhood vaccines were licensed, autism was not considered as a potential side effect, nor were asthma or allergies. Once a vaccine is found to have more benefits than risks, denying children protection is problematic. So studies were done later (and found no link). Or, flu or Tdap vaccines were not initially licensed for pregnant women; that was done later, so pregnant women were not in the clinical trials, and that was done based on other studies.
- Clinical trials are not always ethical or feasible. For that reason, our evidence on cigarettes causing lung cancer is not from clinical trials. That is also why the Institute of Medicine opposed clinical trials of the vaccine schedule that would leave some children unvaccinated.
- And finally, the assumption that clinical trials are always better evidence than other ways of monitoring is problematic. Clinical trials of a drug are done at one point; before licensure. We may have decades of experience with the drug that provides abundant information. That is true of many childhood vaccines. They are extensively monitored, are subject to multiple monitoring systems, and we have abundant data about their safety far beyond clinical trials – and that data led our National Academies of Science and Medicine, our foremost independent institute for science, to conclude that vaccines are safe.
Anti-vaccine “Turtles All the Way Down” summary
In short, in its first chapter, the “Turtles All the Way Down” book repeats a long-standing anti-vaccine trope that there are no placebo trials of vaccines, implying vaccines are not tested. To do so it ignores or misses several clinical trials, misunderstands the difference between later and earlier vaccines, and misrepresents the ethics of testing later and combination vaccines. It also tries to reject all the evidence about vaccine safety beyond the initial clinical trials. There could be more to say – the chapter is 83 pages – but this post is long enough, and I think demonstrates the book’s misuse of sources and errors well enough to warn readers that it is unreliable.
Professor Reiss writes extensively in law journals about the social and legal policies of vaccination. Additionally, Reiss is also member of the Parent Advisory Board of Voices for Vaccines, a parent-led organization that supports and advocates for on-time vaccination and the reduction of vaccine-preventable disease.
