Hierarchy of scientific evidence – keys to scientific skepticism

hierarchy of scientific evidence

I am a scientific skeptic. It means that I pursue published scientific evidence to support or refute a scientific or medical principle. I am not a cynic, often conflated with skepticism. I don’t have an opinion about these ideas. Scientific skepticism depends on the quality and quantity of evidence that supports a scientific idea. And examining the hierarchy of scientific evidence can be helpful in deciding what is good data and what is bad. What can be used to form a conclusion, and what is useless.

That’s how science is done. And I use the hierarchy of scientific evidence to weigh the quality along with the quantity of evidence in reaching a conclusion. I am generally offended by those who push pseudoscience – they generally try to find evidence that supports their predetermined beliefs. That’s not science, that’s the opposite of good science.

Unfortunately, today’s world of instant news, with memes and 140 character analyses flying across social media, can be overwhelming. Sometimes we create an internal false balance,  assuming that headlines (often written to be clickbait) on one side are somehow equivalent to another side. So, we think there’s a scientific debate when there isn’t one.

I attempt to write detailed, thoughtful and nuanced articles about scientific ideas. I know they can be complex and long-winded, but I also know science is hard. It’s difficult. Sorry about that, but if it were so easy, everyone on the internet would be doing science. Unfortunately, there are too many people writing on the internet who think they are talking science, but they fail to differentiate between good and bad evidence.

But there is a way to make this easier. Not easy, just easier. This is my guide to amateur (and if I do a good job, professional) method to evaluating scientific research quality across the internet.

Continue reading “Hierarchy of scientific evidence – keys to scientific skepticism”

Peer review failure – scientific papers retracted

As many of the regular readers of this website know, I put a lot of value in the quality of evidence for a scientific or medical claim, usually in the form of a paper published in a peer reviewed, high impact factor journal. But to be honest, I prefer a paper published in a peer reviewed, high impact fact journal that presents a systematic review of all of the evidence available to provide us with a broad statistical analysis over tens or hundreds of studies. It’s like a peer reviewed paper analyzing peer reviewed papers. Continue reading “Peer review failure – scientific papers retracted”

Despite the meme on Facebook, bananas do not cure cancer

a-ripe-banana-a-day-keeps-cancer-infection-at-bay

This article was published on 29 July 2012, and has had over 70,000 views. This is the number one article I’ve ever written, I enjoyed writing it, but I never thought it would be such a big hit. It basically arose from a meme I saw on Facebook that claimed that bananas with dark spots had anti-cancer compounds in it. And it was all based on a misreading of a published article, a lack of knowledge about tumor necrosis factor, and a complete misunderstanding of human physiology and immunology. And this is my number 1 favorite, and number 1 most popular article for 2013.

 

Note: this article was rewritten and revised–please read and comment on the updated version of this article.

 

Last year, I wrote an article about how to critically analyze pseudoscience and misinformation to get at the scientific evidence which may help you accept or reject something you might read on the internet, even if it appeared to be accurate. On Facebook, Twitter and many internet sites (including Wikipedia), there is an amazing tendency of individuals to accept what is written as “the truth” without spending the effort to determine if what is written is based on accurate science. Twitter, of course, limits itself to 140 characters, which means you either have to click on a link to get more information, or just accept that the 140 characters are factual. And if you can make a complex scientific argument in 140 characters, I’m impressed.

Facebook is filled with false memes on just about everything from politics to medicine. The anti-vaccination crowd fills Facebook with their amusing and highly inaccurate memes. For more than a year, there have been dozens of  photos of bananas with a few words that some Japanese scientists claim that ripe bananas have high levels of “tumor necrosis factor“, so eat bananas to cure cancer and maintain a healthy immune system. Facebook is famous for these things, little pictures with a few words, no sources of the information, and broad conclusions. Eat bananas. Cure cancer. And people share them with a click of the button and move on to the next cute cat picture. It’s really the lazy person’s way of learning. Although who doesn’t enjoy the cute cat pictures? Continue reading “Despite the meme on Facebook, bananas do not cure cancer”

Proliferation of fake peer-review journals

peer_reviewScientific skepticism is the noble pursuit and accumulation of evidence, based on the scientific method, which is used to  question and doubt claims and assertions. A scientific skeptic will hold the accumulation of evidence as fundamentally critical to the examining of claims. Moreover, a true skeptic does not accept all evidence as being equal in quality, but, in fact, will give  more weight to evidence which is derived from the scientific method and less weight to poorly obtained and poorly scrutinized evidence.

In the world of real scientific skepticism, evidence published in a peer-reviewed, high impact factor journal far outweighs evidence taken from other sources. Peer review is the evaluation of a scientific work by one or more people of similar competence (usually in the same field) to the producers of the work. Mostly, the peer review is blinded, in that the reviewers generally don’t know the authors (although it may not be difficult to uncover, especially if the paper is in an esoteric field of science). Peer review constitutes a form of self-policing of science by qualified members of a profession within the field of research. It is through this system of criticism and review that makes many journals, and the articles published within, powerful pieces of evidence in science.

In addition to peer review, there are other ways to ascertain the quality of research in a particular journal. Articles in high quality journals are cited more often because high quality journals just attract the best scientific articles. Higher quality journals employ a more meticulous and exhaustive peer-review.

Although somewhat controversial, journals are ranked using a metric called “impact factor” that essentially expresses numerically how many times an average article in a particular journal is cited by other articles in an index of all other journals in the same general field. The impact factor could range from 0 (no one ever cites it) to some huge number, but the largest is in the 50-70 range. One of the highest impact factor journals is the Annual Review of Immunology, which is traditionally has an impact factor in the 50′s–this would indicate that an average article published in that journal is cited by other medical articles an average of 50 times (an outstanding number). Continue reading “Proliferation of fake peer-review journals”

Checking for pseudoscience in real science news (updated)

One of the larger problems of the internet (OK, there are a lot) is how science is discussed out in the world.  Google any science topic, and you’ll get thousand or millions of hits on any idea in science or medicine. The information is derived from other websites, news reports, rumors, or, to be cynical, from outright fabrication. In the fields of science and medicine, critical thinking is absolutely necessary to understanding it. Because it’s hard work, pseudoscience and anti-science have become quite prevalent lately.   Continue reading “Checking for pseudoscience in real science news (updated)”