In evaluating a scientific claim made by anyone, the only thing that matters is the quality and quantity of evidence. It does matter who is making the claim, it does not matter if you believe their claim, and it does not matter if they make a powerful emotional argument–absent real evidence, it is nothing but words.
When discussing the validity of a scientific or medical claim, some people accept that there is a hierarchy of scientific sources, from nearly worthless (that would be anything from Mercola or Natural News) to scientifically significant systematic reviews. But a lot of people think that if it is published, without any regards to where or how it was peer-reviewed, it signifies the scientific consensus, period, end of discussion. Some will abuse PubMed, the US National Library of Medicine’s powerful search engine, searching for the one article that supports their “beliefs,” while ignoring the 1000 other articles that don’t.
Or how some individuals will use the obscure cell culture study to support their claim that XYZ prevents cancer, while completely ignoring all other evidence that cell culture studies are just an early phase of research, and until it’s confirmed in a large clinical trial with human subjects, the cell culture study barely ranks above conjecture or speculation.
Recently, the scientific journal Nature published a fascinating study ranking the top 100 most cited scientific articles of all time. One of the primary measurements of the quality and importance of a science article is how many times it is cited by others. Being cited implies that the original article provides some important groundbreaking information, its data has been reproduced by others, and/or it establishes a new consensus on a scientific principle.
The number one cited science article of all time, by Lowry et al., described a method to determine the amount of protein in solution. Just about anyone who does protein analysis is familiar with the test (and I used it when I was in grad school)–it has been cited over 300,000 times in the 63 years since it was first published. And despite the passing of six decades since it was published, it’s still cited over 1000 times each year. Personally, I find that incredible.
To me, one of the most interesting findings is that one of the most important biology articles of our time, The structure of DNA by Watson and Crick, is not included in the top 100, because it has been “only” cited 5200 times since it was published over 60 years ago, whereas the 100th on this list was cited 12,000. The Watson & Crick article about DNA forms the basis of genetics, evolution, immunology, biochemistry, and literally dozens of fields of biology–I’m kind of shocked it didn’t fall into the top 100 (I would have thought top 10), but maybe because the article is so well known, no one actually cites it any more. Just a guess.
The top 100 cited articles, based on an arbitrary cut-off point, is really just a trivia exercise at this point. I am not certain if it tells us anything about the most important research ever done by itself. However, the number of citations, by itself, tell us a lot about the quality of scientific research.
I used Thomson-Reuters Web of Science search system (which is not available to the public, it only available by subscription or through an academic library) to search through various articles that had at least 1000 citations, which would tell me indicate to me what were the most important articles, broadly speaking, in science. Again, 1000 citations is purely arbitrary, but it still indicates scientific power of the highest order.
Interestingly, the most cited vaccine article ever was a 2006 New England Journal of Medicine (with Paul Offit, MD as one of the several co-authors) which presented evidence of the safety and efficacy of the rotavirus vaccine, one of the great advances in pediatric medicine of the past few decades. The article was cited over 1300 times, surprisingly, far more than articles by Jonas Salk and the polio vaccine. The huge number of citations for the rotavirus article indicates the importance of both the study and the vaccine. Many of the important vaccine articles generally had several hundred citations, usually 100-200 per year. Not exactly at the level of Lowry’s protein assay article, but impressive nevertheless.
On the other hand, Tomljenovic & Shaw’s article that attempts to make a unsupported causal link between aluminum adjuvants in vaccines and autism, published in a low impact factor predatory journal which has nothing to do with vaccines or neurodevelopmental disorders, has been cited 60 times in the past 5 years–about 12 times a year, and most of the citations were in articles that completely debunked their hypothesis. Yes, someone will tell me that there’s some worldwide conspiracy to suppress the research done by Tomljenovic & Shaw–alternatively, the law of big numbers say that if they were really pushing good science, eventually hundreds of scientists would cite their work. Like Paul Offit.
Recently, I have also been discussing the appeal to authority where an incompetent non-epidemiologist, Peter Doshi, who published an article that trashed the flu vaccine, has been deified by the antivaccination cult, because he happened to publish an article in a real journal. It’s been cited around 3 times over the past 2 years. The top articles about the flu vaccine over the past 15 years all have around 15-20 citations per year. What is really ironic about this level of citation is that most flu articles have short “shelf lives” since the flu vaccine changes annually. Doshi’s article should get more citations because it’s an article about public health policy–yet no one is citing it.
It’s easy to see how many citations a particular article may have by checking Google Scholar, though I do not find it to be as accurate as Web of Science. Annoyingly, both Google Scholar and Web of Science have an odd search system where lots of articles are missed in a search (so I apologize if there’s a more cited vaccine article that I just could find).
Let’s look at what all this means:
- In science, evidence matters. That’s it, no debates, no accusations, no logical fallacies. It’s all about evidence.
- And quality of that evidence matters more.
- And we have the tools, relatively easy to use, that will tell us the quality of that evidence. We can find the Impact Factor or h index of the journal. Journals in the top 50% of either measurement could tell you a lot about the quality of journal. We can see the number of citations (which is more important as time passes).
Too many science deniers misuse search engines to find the publications that support their beliefs, what is called confirmation bias, rather than finding the all the evidence, with a bias to higher quality evidence, to reach a conclusion. Sure I can find bad articles, such as ones by Doshi or Tomljenovic & Shaw, to confirm what I think about vaccines. Or I could spend the time to find all of the systematic reviews, published in top journals and cited hundreds of times, and decide that there’s really strong scientific support for vaccine safety and effectiveness.
If you need to search for accurate and scientific evidence about vaccines try the Science-based Vaccine Search Engine.
Key citations:
- Doshi P. Influenza vaccines: time for a rethink. JAMA Intern Med. 2013 Jun 10;173(11):1014-6. doi: 10.1001/jamainternmed.2013.490. PubMed PMID: 23553143.
- Lowry OH, Rosebrough NJ, Farr AL, Randall RJ. Protein measurement with the Folin phenol reagent. J Biol Chem. 1951 Nov;193(1):265-75. PubMed PMID: 14907713.
- Tomljenovic L, Shaw CA. Do aluminum vaccine adjuvants contribute to the rising prevalence of autism? J Inorg Biochem. 2011 Nov;105(11):1489-99. doi: 10.1016/j.jinorgbio.2011.08.008. Epub 2011 Aug 23. PubMed PMID: 22099159.
- Van Noorden R, Maher B, Nuzzo R. The top 100 papers. Nature. 2014 Oct 29;514(7524):550-3. doi: 10.1038/514550a. PubMed PMID: 25355343.
- Vesikari T, Matson DO, Dennehy P, Van Damme P, Santosham M, Rodriguez Z, Dallas MJ, Heyse JF, Goveia MG, Black SB, Shinefield HR, Christie CD, Ylitalo S, Itzler RF, Coia ML, Onorato MT, Adeyi BA, Marshall GS, Gothefors L, Campens D, Karvonen A, Watt JP, O’Brien KL, DiNubile MJ, Clark HF, Boslego JW, Offit PA, Heaton PM; Rotavirus Efficacy and Safety Trial (REST) Study Team. Safety and efficacy of a pentavalent human-bovine (WC3) reassortant rotavirus vaccine. N Engl J Med. 2006 Jan 5;354(1):23-33. PubMed PMID: 16394299.
- Watson JD, Crick FH. The structure of DNA. Cold Spring Harb Symp Quant Biol. 1953;18:123-31. PubMed PMID: 13168976.
