Checking for pseudoscience in real science news (updated)

One of the larger problems of the internet (OK, there are a lot) is how science is discussed out in the world.  Google any science topic, and you’ll get thousand or millions of hits on any idea in science or medicine. The information is derived from other websites, news reports, rumors, or, to be cynical, from outright fabrication. In the fields of science and medicine, critical thinking is absolutely necessary to understanding it. Because it’s hard work, pseudoscience and anti-science have become quite prevalent lately.  

One only needs to look at the whole vaccine “debate,”  where one group says that “vaccines cause autism,” while the science side says, “there is no evidence for that.” To the uncritical eye, one could accept that both sides are the same, just the debate hasn’t been settled yet. It’s the same with global warming, alternative medicine, creationism, sasquatch, or any other pseudoscience you can find. But in reality, good science journalism will provide you with the real science, whereas bad journalism will still claim it’s a “scientific controversy.” 

Just spend some time on Facebook, you will find lots of misinformation about science and medicine. Just today, someone posted a story on Facebook that some “Japanese scientists” claimed that bananas had high levels of “tumor necrosis factor“, so eat bananas to cure cancer and maintain a healthy immune system. Debunking it was fairly easy:

  • The study was published here (pdf).  There are several issues with it: 1. It’s a mouse model. You cannot believe how many animal models fail to transfer to humans. 
2. The article is published in a low impact journal. This journal has an impact factor of less than 1.0, which indicates a very low quality journal and it’s not even indexed in PubMed, which further indicates its low value. 
3. I dispute the inference that injecting banana extract intraperitoneally can have an inference for ingesting a banana orally. Can we induce the same immune subsystems in the same way? Doubtful.
 4. What is the clinical significance of what was induced? In other words, is there actually TNF isolated from the bananas in a form that actually can do something? Or is there some other effect just because there’s a blob of intraperitoneal banana extract which induces some other immune response. In fact, the paper does not even indicate that the banana itself has any TNF or TNF-like activity in it. 
5. No one has repeated this experiment, and certainly no one has shown this effect in humans.
  • Since tumor necrosis factor (TNF) is a fairly complex protein, with a highly specific role in the human immune system (something notably lacking in a plant), the chances that a banana contains some substance that exactly mimics or copies TNF is so tiny as to be close to impossible.
  • Even if the banana makes TNF, what is the chance that it would survive the digestive tract and be absorbed into the bloodstream? Again, almost impossible, since the digestive tract’s purpose is to break down complex proteins and substances into its constituent components before being absorbed into the bloodstream. The TNF probably wouldn’t survive the digestive tract.
  • Since TNF has a very specific action on the immune response, and not directly on cancer cells, how do we eat sufficient bananas (even if it did have TNF and it could be absorbed into the bloodstream) to increase the level locally to actually cause the appropriate immune response? And which one of the 200 or so cancers would it effect?
  • TNF causes many of the clinical problems associated with autoimmune disorders such as rheumatoid arthritisankylosing spondylitisinflammatory bowel diseasepsoriasishidradenitis suppurativa and refractory asthma. So, if a banana had TNF, if it could pass into the bloodstream from the digestive system, if it could reach high enough levels to actually do something, it probably wouldn’t have any effect on any of the cancers, but it could have serious side effects. However, I wouldn’t worry about it, since there are so many “if’s” that it’s just not possible.
  • It’s clear that the inventor of this meme had no clue that tumor necrosis factor doesn’t mean it kills tumors (it is kind of an odd name). It actually has multiple effects on several parts of the immune response, some of it so complex that some of it can be negative to your health.
  • And this “boost the immune system” myth?  The immune system is an incredibly complex system, with an almost infinite number of interactions between various proteins, compounds, organs, factors, and cells. As long as you’re healthy, so is your immune system, there is nothing you can do to make it stronger, better, or “boostier.” Well, vaccinations help, but they rely upon an appropriate immune response. So even if you won the Nobel Prize by finding some miracle compound that “boosts the immune system,” it would work on one tiny part of that system, and it would have a zero net effect.

Emily Willingham, at Double X Science (great name!), wrote an article, Science, health, medical news freaking you out? Do the Double X Double-Take first, on how to read science or medical news. She lists out six steps on how to analyze the next intriguing science story that you read. This can work on something written in the New York Times, or even just some internet claim that sounds a bit too good to be true. 

  1. Skip the headline. “Headlines are often misleading, at best, and can be wildly inaccurate. Forget about the headline. Pretend you never even saw the headline.” There is nothing to add here. Too often, you’ll see a news article that says “eat this, it’ll cure cancer.” But when you get through the article, it’s more like “eat this, and it may reduce the risk of carcinomas in rats.”  
  2. What is the basis of the article? If it’s an editorial (even in a scientific or medical journal) or a presentation at a conference or meeting, then it is of limited value. Editorials are just opinion, and presentations at meetings lack peer review. Many, if not a majority, of research presented at meetings does not end up being published, and is a forgotten part of the scientific process. There’s a group of “engineers” pretending to push real science, but use pseudoscience based on few peer-reviewed articles, and mostly nonsense meeting abstracts to “prove” their point. Alternative medicine pushers are often well known to “research mine” to find the one presentation that supports their beliefs.
  3. Look at the words in the article. A lot of these news stories contain words like linked, associated, correlation and/or risk. One of the biggest errors one can make is conflating correlation with causation, or the logical fallacy of post hoc ergo propter hoc. In other words, just because one follows another, or one is correlated with another, that is not “proof” of causation. For example, smoking is correlated with lung cancer. That doesn’t prove it causes it. However, there is biological evidence (published in journals) that support smoking’s causative relationship to lung cancer. Humorously, there probably are lot of kids who break their arms soon after getting a vaccination, enough to show some level of mathematical correlation. But stating that the vaccine is the causative factor would be silly, since there just a random number of kids who will break their arms in any period of time, irrespective of the vaccination.
  4. Look at the original source of the information. The gold standard of science is a review article from a respected journal. The Cochrane reviews are one of the best (but far from perfect) source for meta-analyses of numerous publications in medicine. They are the basis of evidence-based medicine by rolling up all of the published evidence, detecting flaws in some studies, and coming to a conclusion. A single published article should be taken with grain of salt, even if published in a great journal, until there is further confirmation and repetition of the experimentation, which is why a review article is so valuable. The next highest standard of scientific publication is a peer-reviewed paper in a high-impact journal. Peer-review is a simple concept–experts anonymously review the submitted article to determine if it’s well written, brings new information, and seems plausible. It’s not magic. It’s not a conspiracy of scientists to keep out information, it is just a method to make sure the article and science is plausible and worthy of inclusion. A high-impact journal is one that has high standards, is well-read, and is cited frequently by other articles. Science, Nature, the Proceedings of the National Academy of Sciences, the British Medical Journal, and others are high impact factor peer reviewed. The quality drops from there. Abstracts or presentations from a conference or meeting are not peer-reviewed and are mostly speculative. That does not mean they should be dismissed absolutely, but if they’re bringing out something new, it has a while before it moves up to the higher level of publication. The worst of the worst are press releases from any organization (usually the university where the researcher is located) or articles published without any type of peer review. Any news article should have links back to the original peer-reviewed article, or make it easy to find that article
  5. Remember that every single person involved in what you’re reading has something to gain. The news outlet needs clicks to get more advertising dollars. The reporter needs clicks to keep their job at the news outlet. Some individuals purporting to present real medical news, like Joey Mercola, are just trying to sell potions that have no meaningful effect. Of course, a press release from a respected university, pharmaceutical company, or research institution is just trying to get fame, money and prestige. In other words, be skeptical of everything, until you can confirm it. Skepticism, as opposed to denialism, insists that you keep accumulating evidence to support or debunk a claim. 
  6. Ask a scientist. Scientists love questions. They love answering questions. They love teaching. Neil deGrasse Tyson, probably one of the most popular astronomers since Carl Sagan, will answer questions asked of him on Twitter or Facebook. Will he answer nonsense questions like “why do you hate Pluto so much?”–probably not. But maybe he’ll give a reasonable answer to “what’s the chance of an asteroid hitting earth in the next 50 years?”  Jerry Coyne, one of the top evolutionary biologists has a website.  Richard Dawkins has a website too. If you want to chat with some of the best scientists in medicine, Paul OffitSteven Novella, and David H. Gorski, you will find that all are open to being contacted and will probably answer your questions. Many pseudoscientists, like Ken Ham or the Age of Autism, refuse to answer questions with science, resorting to anti-scientific rhetoric, or usually just delete comments that don’t meet their simple-minded dogma. If you want real science, there are many scientists on Twitter and Facebook that are easy to find. By the way, since so much time on this blog is spent on vaccines and the non-existent link to autism, Liz Ditz’s blog should be a first stop for more information. She is much more civil than most of the “vaccine is safe” crowd.

Real science is open to the bright lights of analysis and criticism. It doesn’t try to hide anything, so one must take the time to dig for the actual information. Read any global warming denialist news piece, and it will base it’s information on opinion pieces or a letter written by a few NASA engineers claiming global warming doesn’t exist. And the denialist news article will never refer to peer-reviewed articles in respected science journals.  

Well written scientific journalism should always point you to the essential research. Bad journalism makes you work to find the actual science, but if you persevere you will find the necessary information to make an informed and critical analysis. Maybe if you do, you won’t be eating 100 bananas to get your dose of tumor necrosis factor.

(Note: a further update to analyzing published scientific sources can be found here.)

via Combatting A Tsunami of Pseudoscience: The Double X Double-Take – I Speak of Dreams

The Original Skeptical Raptor
Chief Executive Officer at SkepticalRaptor

Lifetime lover of science, especially biomedical research. Spent years in academics, business development, research, and traveling the world shilling for Big Pharma. I love sports, mostly college basketball and football, hockey, and baseball. I enjoy great food and intelligent conversation. And a delicious morning coffee!

  • Pingback: آيا واقعا موزهاي خيلي رسيده و لك دار خواص ضد سرطاني دارند؟ | دانستنيها()

  • Ettina

    “Humorously, there probably are lot of kids who break their arms soon after getting a vaccination, enough to show some level of mathematical correlation. But stating that the vaccine is the causative factor would be silly, since there just a random number of kids who will break their arms in any period of time, irrespective of the vaccination.”

    If it’s truly random, there won’t be any correlation.

    A better analogy would be that many kids lose all their baby teeth within a few years after starting school. There probably is a correlation between the two, but we know going to school doesn’t make you lose your baby teeth. They are simply both caused by the same thing – being five years old.

  • Pingback: Web Skepticism Tutorial - Fuel Economy, Hypermiling, EcoModding News and Forum -

  • Pingback: خواص ضدسرطانی موز با لکه‌های تیره چقدر صحیح است؟! | گُمانه()

  • Pingback: Ripe bananas do not prevent cancer()

    • Ettina

      Mice are not humans. They share some traits in common with us, but others are vastly different.
      For example, most mammals, including mice, produce their own vitamin C. Humans don’t – we get our vitamin C from our diet. So scurvy can affect humans, but not mice.
      Other things are shared, of course. For example, the impact of maternal licking and grooming on a mouse pup’s brain development is remarkably similar to the effect physical contact with a caregiver has on a human infant’s brain development. The physiology of a mother’s love is similar in both species, too. As a result, mice make a good model animal for the effects of child abuse and neglect, especially with regards to anxiety and cortisol regulation.

    • Ettina

      Just because a child was normal and became autistic soon after getting a vaccine doesn’t mean the vaccine caused their autism. It could have just happened to occur at the right time.
      For example, there is a genetic condition known as Rett Syndrome, caused by a particular kind of mutation in the MECP2 gene (located on the X chromosome). The mutation is lethal in boys, and causes Rett Syndrome in girls. Like kids with regressive autism, girls with Rett Syndrome develop normally at first. Between 6 to 18 months, the girl’s head growth and cognitive development will start to lag behind other children, and then, they start to regress. They lose speech, hand skills, and in some cases the ability to walk, and then the regression stops, leaving the child severely disabled for the rest of their life.
      We see the effects of timed developmental-regulation genes most clearly in physical development – things like cutting first teeth, losing baby teeth, entering and completing puberty, etc. All of those are genetically programmed to occur at a certain age. Well, there are brain changes that are genetically programmed as well.
      If anything goes wrong with the genes involved in timing of developmental processes, you get a condition that will only activate in a certain stage of development. In the case of Rett Syndrome, it’s neuronal pruning gone overboard. A 6 month old baby has way more neurons than they’ll need as an adult – in fact, they have too many neurons too function optimally. So, as they learn new skills, they prune unused neurons, while keeping the ones that are in use. But the brain of a girl with Rett Syndrome is not able to restrain the neuronal pruning process effectively, so it prunes neurons that are in use as well as ones that are not being used. This results first in developmental slowing, and then in regression. Then, when the neuronal pruning process reaches it’s scheduled end point, the regression stops.
      We don’t yet know what causes regression in autism. However, it’s entirely possible that it’s caused by a genetically programmed neuronal process gone haywire, just as it is in Rett Syndrome. Most kids get vaccinated at the same ages – eg getting the MMR jab at 18 months instead of at 12 months or at 36 months. So, if a genetic condition causes some children to regress starting around 18 months, many of them will regress right after getting the MMR jab.
      Certainly, there are kids who have been harmed by vaccines. Hannah Poling, for example, had an undiagnosed mitochondrial disorder. As a result, when she experienced a mild fever after a vaccination and lost her appetite, neither her parents nor her doctors realized that she would suffer a metabolic collapse if they didn’t make her eat within 24 hours. This metabolic collapse lead to a brain injury which caused symptoms of autism. However, cases like hers are rare – far rarer than autistic regression is.

  • Pingback: Are bananas really able to protect against cancer? « Bad News Biology()

  • Pingback: GMO opponents fall for a hoax()

  • Pingback: Another silly internet meme–bananas and tumor necrosis factor()

  • Pingback: Harvard hospital retracts statement about data on aspartame and cancer()

    • Pseudoscience always attempts to bolster its claims with some real science. However, the Japanese studies falter from many deficiencies:

      1. It’s a mouse model. You cannot believe how many animal models fail to transfer to humans. I forget the exact percentage, but if I recall correctly, it’s below 10%.
      2. The article is published in a low impact journal. This journal is <1.0, which indicates a very low quality journal.
      3. I dispute the inference that injecting banana extract intraperitoneally can have an inference for ingesting a banana orally. Can we induce the same immune subsystems in the same way? Doubtful.
      4. What is the clinical significance of what was induced? It could be bad.
      5. No one has repeated this data.

      Color me skeptical.

  • Meaning?

  • Pingback: Naturopathy and diabetes–pure pseudoscience()

  • Pingback: Pseudoscience and the desperate anti-vaccine intrigue()