I always get suspicious when someone makes an argument with the statement of “it’s been proven to work”, “the link is proven”, or, alternatively, they state some negative about scientifically supported therapies. Typically, I hear these kinds of statements from the pseudoscience pushing crowd. For example, real science has debunked the “there is a proven link between vaccines and autism,” a common and popular pseudoscientific belief. Or that most alternative medicine (CAM) therapies work based on numerous logical fallacies that suspends reason, and accepts “belief” in the therapy, something that evidence-based medicine just doesn’t do.
So, I decided to put together a rather substantial treatise on science vs. pseudoscience. We’ll explore what exactly makes an idea scientific (and spoiler alert, it isn’t magic), and contrary to real science, what makes an idea “pseudoscientific.” So sit down, grab your favorite reading beverage, because this isn’t going to be a quick internet meme. I intend to show you exactly how pseudoscience, whether it’s creationism, vaccine denialism, alternative medicine, or whatever you want to debunk, lies. Yes, lies.
The scientific method
Science rarely uses the term “proven”, because the scientific method is not a system to make a definitive answer on any question–scientists always leave open the possibility of an alternative hypothesis that can be tested. If the alternate hypothesis can be supported through experimentation, then it can replace the original one. When an alternative medicine or junk science supporter states “it has been proven,” ask where is the evidence. What is more troubling is that someone who believes in these therapies cannot imagine that they don’t work, what is called falsification, which is a hallmark of good science. Whenever I hear that a scientist say, “we were wrong, it doesn’t work,” my retort is “excellent, good science.”
The scientific method is an unbiased systematic approach to answer questions about the natural world, including medicine. It has several basic steps:
- Define the question–this could be anything from “does this compound have an effect on this disease?” or “how does this disease progress?”
- Observations–this is the subjective part of science. Do we observe trends or anomalies? Does a physician notice that every patient from a town or neighborhood exhibit the same disease? A lot of science arises from observations of the natural world, and yes, some of those observations can be anecdotes or personal observations. For example, one of the most famous stories in the early history of medicine is when Edward Jenner observed that milkmaids rarely were infected by smallpox because they were exposed to cowpox, a less virulent disease.
- Hypothesis–using the observations, create a hypothesis that can be tested. In Jenner’s case, he hypothesized that exposure to cowpox immunized individuals to small pox.
- Experiment–simply, the scientist then tests the hypothesis with experiments and collects the data. The experiments are not designed to solely validate the hypothesis but may also attempt to refute it. In real science, attempting to nullify one’s own hypothesis is an honorable pursuit.
- Analyze–examining the results carefully, usually using acceptable statistical methods.
- Interpret–sometimes the data leads to a revision of the hypothesis, which means the scientist has to return to steps 3-6. Or it confirms or supports the hypothesis, which means the researcher can move to Step 7.
- Publish–in today’s scientific community, scientific data and analysis is subject to the scrutiny of other scientists before it can be published, a process called “peer-review.” This is a critical step that ensures that the results can stand up to criticism of others.
- Retesting–Many times the research is repeated by others, or the hypothesis may be slightly revised with additional data. Science is not static, it constantly revises theories as more data is gathered. For this reason alone, science is not an absolute, it is constantly seeking new data.
Science is an evidence-based systematic analysis without inherent opinion or emotion. In other words, it is a method to cut through opinions and anecdotal observations, so that one can have some reasonable expectation that a medicine or device will work as planned, or if a theory can be predictive.
Complementary and Alternative Medicine (called CAM), for example, fails to utilize scientific method. Supporters of CAM usually perform experiments to confirm their hypotheses, never attempting to refute it, because the attempt to refute may provide you more solid evidence than attempting to confirm. This is the fundamental principle of falsifiability, that is, that if a hypothesis is false, it can be shown in experimentation that allows science to have an open mind about the world. When you speak to a believer of CAM, they almost never assume that their treatment cannot work.
It’s interesting that CAM and pseudoscience start out with observations of the real world. For example, CAM therapies sometimes show positive results, not because of the therapies themselves, but because humans just get better from many diseases. So, these CAM therapies rely upon Confirmation Bias, that is, the tendency to accept information that supports your beliefs, or even Post hoc ergo propter hoc, a logical fallacy which says “since that event followed this one, that event must have been caused by this one.” Humans too often conflate correlation and causation. Just because events follow one another, that doesn’t mean one causes the other. I suppose that’s how superstitions arise.
The “pseudoscientific method”
To identify pseudoscience, there are six reliable clues that shout out “pseudoscience.” Almost always, you can find all six in any pseudoscientific claim.
- Use of vague, exaggerated or untestable claims. Essentially, pseudoscience makes “scientific claims” that are vague and variable, rather than the precise scientific claims made with statistical analysis, usually a hallmark of the specificity associated with scientific research. Pseudoscience also fails to make use of operational definitions of variables, terms and other information so that an independent researcher can repeat or test the claims. Pseudoscience lacks boundary conditions (well-supported scientific ideas possess carefully described limitations under which the prediction of results do and do not apply); lacks effective controls, such as utilizing placebos and double-blind experiments; and lack understanding of basic and established principles of physics and engineering. Importantly, pseudoscience suffers from the absence of parsimony, sometimes referred to as Occam’s razor, that is, failing to seek an explanation that requires the fewest possible additional assumptions when multiple viable explanations are possible. Finally, it uses obscurantist language, and use of apparently technical jargon in an effort to give claims the superficial trappings of science.
- Extreme reliance on confirmation rather than refutation. Pseudoscience relies upon assertions that do not allow the logical possibility that they can be shown to be false by observation or physical experiment, a concept called falsifiability. In other words, real science assumes that there could be an experiment designed that would refute (or nullify) the hypothesis. Pseudoscience also asserts claims of predictability when it has not been shown to be predictive; “scientific” claims that do not confer any predictive power are considered at best “conjectures”, or at worst “pseudoscience”. Pseudoscience also maintains that claims which have not been proven false must be true, and vice versa, which also known as the Argument from Ignorance. Pseudoscience is also over reliant on testimonial, anecdotal evidence, or personal experience. Yes, this type of evidence can be used as observations to develop a hypothesis. But they cannot be used to test the hypothesis itself. Pseudoscience also overuses confirmation and selection bias in the presentation of data that seems to support its claims, in the meantime, suppressing or refusing to consider data that conflict with its claims. In real science, the burden of proof rests on those making a claim, not on the critic. Pseudoscience employs the reversed burden of proof, demanding that skeptics demonstrate beyond a reasonable doubt that a claim, for example, that some supplement prevents cancer, is false. It is essentially impossible to prove a universal negative, so this tactic incorrectly places the burden of proof on the skeptic rather than the claimant. Finally, pseudoscience appeals to holism as opposed to reductionism: proponents of pseudoscientific claims, especially in alternative medicine, homeopathy, naturopathy and mental health, often resort to the “mantra of holism” to dismiss negative findings.
- Lack of openness to testing by other experts. Pseudoscience researchers evade peer review before publicizing results, occasionally using press conferences to share their ideas. These “researchers” will claim that their ideas contradict the scientific consensus, so they must avoid the peer review process because that process is biased towards the established paradigms and consensus. They will also claim that their results cannot arrive from the scientific method. Thus, they get to avoid the feedback of informed colleagues. They will also appeal to the need for secrecy or proprietary knowledge when an independent review of data or methodology is requested. Of course, many agencies and institutions that fund real science research require authors to share data so it may be evaluated independently.
- Absence of progress. Pseudoscience usually fails to progress towards providing or even searching for additional evidence of its claims. Astrology is an example of a pseudoscientific concept that has not changed in 2000 years. Real science is constantly adding data through scientific progress. Moreover, even when pseudoscience attempts to find evidence, the statistical significance of those experimental results do not improve over time and are usually close to the cutoff for statistical significance. Normally, experimental techniques improve as the experiments are repeated, and this gives ever stronger evidence to a scientific principle or hypothesis. If statistical significance does not improve, this typically shows the experiments have just been repeated until a success occurs due to chance variations.
- Personalization of issues. Pseudoscience is often composed of closely tied social groups, and usually includes an authoritarian personality, suppression of dissent, and groupthink. This social construct can enhance the adoption of beliefs that have no rational basis. In an attempt to confirm their beliefs, the group tends to identify their critics as enemies. They also make assertions of claims of a conspiracy on the part of the scientific community to suppress results that support the pseudoscience. Finally, they attack the motives or character of anyone who questions the claims, the Argumentum Ad Hominem. As an example, the anti-vaccine crowd has invented numerous claims about Dr. Paul Offit, one of the great researchers in vaccines, just to attack him personally. They’ve done the same with Bill Gates.
- Use of misleading language. They try to create scientific-sounding terms to add weight to claims and persuade non-experts to believe statements that may be false or meaningless; for example, a long-standing hoax refers to water by the rarely used formal name “dihydrogen monoxide” and describes it as the main constituent in most poisonous solutions to show how easily the general public can be misled. More often, pseudoscientists use established technical terms in idiosyncratic ways, thereby demonstrating unfamiliarity with mainstream work in the discipline.
The differences between science and pseudoscience
There are several types of logical fallacies that the pseudoscience crowd employs. My favorites are Appeals to Antiquity or Tradition, or old ideas are somehow better than new ideas; Appeals to Authority, or someone who should know better supports the claim even if everyone else does not; Bandwagon Fallacy, or everyone does it, so it must be useful; and the Genetic Fallacy, where the source is matters more than the merits of the evidence. I’ve created my favorites list of logical fallacies on this website.
The typical pseudoscientist will use logical fallacies to state very definitively that “it’s proven.” It’s the same whether it’s creationism (the belief that some magical being created the world some small number of years ago), alternative medicine (for example, homeopathy, which is nothing but water, has magical properties to cure everything from cancer to male pattern baldness), or vaccine denialists. The worst problem is that in the world of the internet, if you Google these beliefs, the number of websites and hits that seem to state that they are THE TRUTH™ overwhelm those that are more skeptical or critical.
So how can you tell the difference between science and pseudoscience in medicine? In medicine, we gather and analyze evidence in one of two ways.
Almost any medical product, device, drug or procedure must, by law, be tested in a Randomized Controlled Trial, sometimes called a clinical trial. Essentially, it is a scientific experiment, designed to test the hypothesis of whether the safety and efficacy of a particular medical product is better than a placebo. That is, does the medical product produce results better than doing nothing at all? This is the “gold standard” of investigation, and if the study does confirm the hypothesis, you can be assured it has a benefit to your life (although the degree may be subject to argument).
Alternative medicine just doesn’t do this (most of their reasoning is that their beliefs just doesn’t fit into the clinical trial model), so there is no proof that their products work. A clinical trial usually has thousands of participants, and is done in a manner that the patient and the physician do not know who is and who is not receiving the treatment. The results are analyzed statistically and published in peer-reviewed journals. Furthermore, the results are reviewed and investigated by the FDA (and legal bodies in other countries), before a drug or device can be used by a physician. This is an expensive and time-consuming process, in which alternative medicine hardly ever participates.
Now it’s not ethical to test every medical hypothesis with a clinical trial. For example, we know that smoking is bad for your health. Yet, tobacco manufacturers love to insist that there has never been a clinical trial that makes this conclusion. The reason that is true is that it would be unethical to give one group of adults cigarettes for 20 years and another group nothing to see if one would die at a higher rate. So we use epidemiological studies to determine if we can see in a population whether a cause has an effect. We can review records of thousands of smokers to see what the effect will be.
Epidemiology is a scientific process that is critical to preventative medicine–without it, we cannot know if some behavior or public health issue has a causal effect on health. Vaccine deniers also love to play with clinical trials by inventing outrageous and highly immoral clinical trials for vaccines. For example, they would like kids to be place in a placebo group vs a vaccine group to see if there are any adverse effects or if even if the vaccine works.
Remember, anecdotes (“my mother’s friend’s cousin’s daughter was cured by eating this leaf”) are not reasons to accept alternative medicine. Even anecdotes that try to sound like science (“90 out of 100 people think this leaf does work”) aren’t a reason to “believe” in a pseudoscience. And having a high number of anecdotes does not equal “data.”
You might have heard that taking lots of Vitamin C helps prevent colds. It doesn’t–that conclusion is supported by large clinical trials. So unless you are afflicted by scurvy, there’s no reason to take large doses of the vitamin. And that’s the difference between real science and pseudoscience.
Pseudoscience vs evidence (or science) based medicine
In an outstanding analysis of pseudoscience, Steven Novella, MD, a clinical neurologist at Yale University, illustrates how pseudoscience (those who pretend to praise the scientific method, yet do it in a way that is not actually science) and anti-science (those who repudiate science outright, or even undermine science, with subjective analysis and untestable spirituality) is used to reject evidence-based medicine:
This leads us to the final continuum – the consensus of expert opinion based upon systematic reviews can either result in a solid and confident unanimous opinion, a reliable opinion with serious minority objections, a genuine controversy with no objective resolution, or simply the conclusion that we currently lack sufficient evidence and do not know the answer. It can also lead, of course, to a solid consensus of expert opinion combined with a fake controversy manufactured by a group driven by ideology or greed and not science. The tobacco industry’s campaign of doubt against the conclusion that smoking is a risk factor for lung cancer is one example. The anti-vaccine movement’s fear-mongering about vaccines and autism is another.
Basically, science evolves over time. A conclusion that lacks sufficient evidence may eventually be supported by better analysis or groundbreaking research. You’ll notice that anti-science and pseudoscience pushers do not allow themselves to participate in the this continuum of research–they state emphatically that they are right.
Science, by its very nature, must be falsifiable, meaning that any hypothesis or theory has the logical possibility that it can be contradicted by an observation or the outcome of a physical experiment. Just because a hypothesis or theory is “falsifiable,” we do not conclude that it is false. To the contrary, we understand that if it is false, then some observation or experiment will provide a reproducible result that is in conflict with it. Simply put, science assumes that it has it all wrong, and attempts to determine why a particular theory or hypothesis is wrong. Of course, in these attempts, usually more evidence is found to support the original theory. Just because science requires falsifiability, that does not mean that it will ever be falsified, but science is open to the possibility. In other words, science evolves.
Pseudoscience, by its very nature, is not falsifiable. It is mostly based on assertion rather than scientific observation, so it cannot be tested by experiment or observation. Creationism is a perfect example. It is based on a human text (the bible), so there is no experiment that could be designed to test the text, since it non-responsive in a natural sense. It would be like trying to scientifically show that the muppets existed.
Clearly, science depends upon evidence, and in fact, values evidence above all other information. Pseudoscience depends upon faith or belief in what is being pushed. It actually devalues evidence.
Two of the most misused and misunderstood terms in evaluating scientific evidence are correlation and causation, two powerful analytical tools that are critical to evidence based medicine. Correlation is the grouping of variables that may occur together. For example, smoking correlates with lung cancer in that those who smoke tend to develop lung cancer at a statistically significant rate. It’s important to note that correlation does not prove causation. However, once you have numerous well-designed studies that correlate lung cancer to smoking, along with adding in biological and physiological models that support the correlation, then we can arrive at a consensus that not only is smoking correlated with lung cancer, it causes it.
We observe correlations every day. But they are subjective observations for which we cannot state a causal relationship without substantial research. The anti-vaccination movement is rife with these observations which they use to “prove” a cause. An anti-vaccine conspiracy website claims that pregnant women are miscarrying babies after getting the shot. The fact is that there is a statistical chance that women will miscarry during any pregnancy. This is random variability not a cause. In fact, based on the rate of miscarriage, we could expect that thousands of women would miscarry within 24 hours of getting the H1N1 flu shot. But it’s not correlation, unless significant studies show a causal relationship. For example, I’m also sure that thousands of people broke a bone or had a desire to eat a burger after getting the shot, but that’s because in a large enough population of individuals, you can find literally millions of different actions after getting a shot.
So, the miscarriage rate after receiving the swine flu shot is not correlated. It’s just a random observation. And there is no biological cause that could be described. Nevertheless, the “flu vaccine causes miscarriage” conspiracy has been thoroughly debunked by research, but still the internet meme continues. Pseudoscience sometimes uses the same methodology (or lack of methodology) to make positive assertions. Homeopaths will claim that their dilutions will cure whatever disease, yet they do not have scientific evidence supporting them, but there plenty of evidence that debunks what they practice.
As part of my analysis of medical claims of causation or “cures”, I often use this logic to test the possibility of the usefulness of any alternative medicine–is there any physical, chemical or biological mechanism that will allow the quack procedure to work? If you cannot imagine it without violating some of the basic laws of science, then we should stand by Occam’s razor, which states often times the simplest solution is the best. So, if there is no evidence of vaccinations being correlated, let alone causal, to autism, then that remains the simplest solution. To explain a possible tie without any evidence would require us to suspend what we know of most biological processes.
As I’ve said in other posts, the internet gives us so much information, we tend to value it equally, as if every website provides accurate and logical data points. Maybe you have a friend who had a miscarriage 24 hours after receiving the swine flu vaccine. Maybe you’ve heard that many people have. But that’s not science, that’s just a subjective observation. Or even confirmation bias.
Once again, Dr. Novella says it perfectly:
In conclusion, correlation is an extremely valuable type of scientific evidence in medicine. But first correlations must be confirmed as real, and then every possible causational relationship must be systematically explored. In the end correlation can be used as powerful evidence for a cause and effect relationship between a treatment and benefit, or a risk factor and a disease. But it is also one of the most abused types of evidence, because it is easy and even tempting to come to premature conclusions based upon the preliminary appearance of a correlation.
One more thing
Let’s talk about skepticism. Let’s be clear what it is, because the term is abused by so many. Real scientific skepticism is is the noble art of constantly questioning and doubting claims and assertions, and holding that the accumulation of evidence is of fundamental importance. It forms part of the scientific method, which requires relentless testing and reviewing of claimed facts and theories. In other words, skepticism is not just the questioning or doubting of claims, it is gathering all of the evidence and weighing it properly.
A real skeptic, for example, gives more weight to evidence provided in a meta-review articles (which rolls up all of the evidence for and against a particular hypothesis, removing obviously biased or poorly done studies) than to anecdotal commentary from an anonymous website. A real skeptic will value evidence that comes from people who are experts in the particularly field even over those who claim an expertise based on education or research in an unrelated field. An immunologist knows more about immunology than a biochemist, because it takes so long to understand a field of science.
On the other hand, there are those who claim to be skeptics, but are, in fact pseudoskeptics. A pseudoskeptic, in its proper use, is someone who will claim they are a skeptic of a concept, but in reality would not be convinced by any evidence that might be presented. They are close-minded to any evidence, or their standards of evidence are such that it approaches the Nirvana Fallacy, or they want evidence that’s perfect otherwise, it’s worthless. Sometimes, a pseudoskeptic will often call a real skeptic a “pseudoskeptic”, but that’s just a misuse of the term, a misdirection to confuse the argument. Pseudoskeptic is clearly a word that is a synonym of denialism, as there is usually a vast amount of real evidence which is simply willfully ignored by these pseudoskeptics.
For example, there is an individual on Facebook, Vaccine Skeptic Society, who is a proven vaccine denier and a pseudoskeptic. Real skeptics are always prepared to change their positions based on new evidence, relying on the scientific method. Vaccine Denier Society (to use proper terminology) has been shown to ignore all evidence except that which support their position, which isn’t skepticism. It’s just blindly using confirmation bias and ignoring evidence, including the quality of that evidence.
Don’t fall for the trap that you should be “open-minded” or neutral to anti-science or pseudoscience. Open-mindedness and neutrality are expectations that you will balance real scientific evidence, not treat the rhetoric as if it has equal weight to scientific method.
The other day I engaged in a discussion with a well-meaning pharmacist who claimed that vitamin C could treat colds and prevent other diseases like “cancer” (I always get annoyed by people who lump all cancers together, when there are over 200 different cancers, all of which have different histologies, genetics, and causes). Of course, there are a number of well-controlled clinical trials that show that vitamin C has no effect on the common cold and has no effect on cancers so far studied (see this, this, this and this).
During our discussion, she said “science should be neutral,” with the implication that I wasn’t neutral. I happen to agree, science should be neutral and it should balanced. But science should be based upon the scientific method not rhetoric and not using science-y words, at which the writers of Star Trek were well-versed.
So, when I say that I am (or any evidence-based real scientist), “scientifically neutral”, that means I’m willing to weigh the evidence and publications on real scientific issues. I am not a geologist nor a paleontologist, but I was studying biochemistry when the earliest theories on what caused the Cretaceous-Tertiary extinction event 65.5 million years ago, where all the dinosaurs (well, not all, since technically modern birds are surviving dinosaurs from a cladistic point of view) and some 75% of all extant genera died out or were killed. In 1980, Luis Alvarez, a brilliant theoretical physicist whose career was extraordinary, along with his son, Walter Alvarez (a geologist), and Frank Asaro, published an article that describe a sedimentary layer across the world that included a rare element, iridium, usually found in extraterrestrial objects like meteors and comets. The layer fell right at the geological boundary layer that defines the extinction event, so they proposed that the dinosaurs were wiped out by a huge comet or asteroid.
Today, the bolide impact is accepted as a scientific theory with tons of confirming evidence (although some scientists aren’t fully convinced, they still propose some other natural event). But at the time, it was ridiculed, and scientific meetings were quite energetic in discussions. The point is that the impact theory displaced the prevailing scientific theory of the extinction event (well, there were several). One theory replaced another not by rhetoric or appeals to antiquity or belief or faith or pseudoskepticism or anything else. In fact, in this story, one scientific hypothesis was replaced by another and eventually developed into a solid theory.
We should be neutral in science, first reviewing the evidence, next objectively valuing the quality of the evidence, and finally using that evidence to support and refute hypotheses. We did that with the impact event for the dinosaur extinction. We’ve done that with the safety and efficacy of vaccines. Science does that all the time.
Using scientific neutrality, vitamin C doesn’t work, not because I have an opinion, but because there is no science supporting its efficacy. There are some articles that support its use in preventing or treating colds, or megadoses for curing cancer, but those articles are in low-impact (meaning obscure) journals or with poorly designed, or non-blinded trials. My friendly neighborhood pharmacist was stating that I should be neutral between real science and well, nothing at all. That’s not balance, that’s a close-mindedness to science and the scientific method. That’s using opinion as the balance to scientific knowledge. The only thing that should balance scientific knowledge is more scientific knowledge.
Pseudoscience is easy. It doesn’t take work. It’s the lazy man’s (or woman’s) “science.” But it has no value, and because it lacks high quality evidence in support of it, it should be dismissed, and it should not be a part of the conversation.
And one more thing. Real science is hard. You don’t get to be an expert in biochemistry, astrophysics, immunology, multivariate mathematics, or any other natural science just by surfing the internet for a few hours or days. Real science is sitting in dozens of classes over years absorbing and understanding the decades of research that preceded you. It means learning how to be critical, not for criticism’s sake, but to find a new idea that might blossom in to the next best thing in science. It means spending years of your life studying a small idea. It means being smarter than almost anyone else you know. It means writing better than any of your friends or pals. It means late nights and early mornings. It means standing up to the criticism of your peers and of the leaders in your field. And if you put in the really really really really hard work that it takes to be a scientific expert, then you will be given due weight to your ideas. And maybe you’ll change things.
But if you’re going to be lazy, and tell me that vaccines don’t work because “I know my friend’s children got sick after being vaccinated”, at best I’m going to dismiss your pseudoscience. At worst, I’m going to call you a liar and give you my full onslaught of mockery and insults. You deserve no better.
I respect real skeptics. But using the claim of skepticism to support a pseudoscience–that is simply nonsense, and that’s that.