The anti-lockdown COVID-19 deniers are pushing a study from the respected Johns Hopkins University because it seems to indicate that lockdown rules didn’t work. Once you dig into the study, you find that there’s not much there. Of course.
I have this rule that whenever someone quotes evidence by saying it’s from the “Harvard study,” or “Yale study,” or “Johns Hopkins study,” I tend to turn up the sensitivity of my skeptical radar because they’re employing the Argument from authority. In this case, the authority is a respected research institution.
However, as I keep saying, it’s not who or what says something, only the evidence matters. And in this case, even though this “Johns Hopkins lockdown study” seems legitimate, once you did into the study, you will find that there is no evidence whatsoever supporting their claims. It’s a terrible study.
What is the “Johns Hopkins lockdown study?”
The paper is a “working paper” that has not been peer-reviewed or published by any respected medical journal. In fact, it was published on the website of the Institute for Applied Economics, Global Health, and the Study of Business Enterprise at the Johns Hopkins Krieger School of Arts and Sciences in Baltimore. This was not published on the website of the highly respected Johns Hopkins medical school or by experts in the departments of public health or epidemiology. But we will get to the criticism a bit later.
The title of the article, “A literature review and meta-analysis of the effects of lockdowns on COVID-19 mortality,” makes it appear to be a systematic review or meta-analysis, which I consider to be the pinnacle of the hierarchy of biomedical research. Of course, that does not mean that these types of studies are always right, one still has to look at the paper and determine whether it was done well or not.
The authors reviewed 34 papers (more on that later) to conclude that lockdowns in Europe and the United States only reduced COVID-19 mortality by 0.2% on average. I hate when people think that reducing mortality by 0.2% is a bad thing. But is it really 0.2%? We’ll get to that.
The authors concluded:
While this meta-analysis concludes that lockdowns have had little to no public health effects, they have imposed enormous economic and social costs where they have been adopted. In consequence, lockdown policies are ill-founded and should be rejected as a pandemic policy instrument.
But did they really find that? Time for my takedown of this article.
Critique of the study
I’m going to bullet point my criticisms of this study, just to make it easier to read and digest. I hope you don’t mind.
- Peer review. The article was not peer-reviewed by any experts. It is essentially self-published on a website run by the authors. By itself, it does not mean anything, but if you’re going to claim that this is a “study,” then publish it.
- Authors’ credibility. The study’s author Steve Hanke, PhD, is the founder of the Institute for Applied Economics, Global Health, and the Study of Business Enterprise. Dr. Hanke is an applied economist, not an epidemiologist, public health expert, infectious disease expert, or medical doctor. Dr. Hanke is also a senior fellow at the Cato Institute, a libertarian think tank, that has been against vaccines and COVID-19 lockdowns. Hanke’s co-authors are Jonas Herby, MS, a “specialist consultant” at the Center for Political Studies in Copenhagen, and Lars Jonung, PhD, professor emeritus of economics at Lund University in Sweden — neither Herby or Jonung are epidemiologists, public health experts, or medical doctors.
- More on the authors’ credibility. According to Gideon Myerowitz-Katz, PhD, a real epidemiologist at the University of Wollogong in Australia, the authors are “highly regarded economists who have also been extremely anti-lockdown since March 2020.
- Definition of lockdown. They had a very broad definition of lockdown. They consider something as simple as a mask mandate to be equivalent to a lockdown. For most of us, a lockdown is something a lot more, like stay-at-home orders that were prevalent during the early part of the pandemic. Since their definition was so broad, they included disparate studies which is the antithesis of a good systematic review.
- Quality of included studies. A good systematic review or meta-analysis is a secondary study that rolls up the data from other high quality studies. Part of the review process is to include or exclude studies based on strict criteria in the hopes of removing bias. According to an article in Forbes, of the 34 papers ultimately selected to be included in the systematic review, 12 were “working papers” rather than peer-reviewed science. And 14 studies were conducted by economists rather than public health or medical experts.
- More about that quality. Dr. Meyerowitz-Katz stated that the review did not include “modelled counterfactuals…the most common method used in infectious disease assessments” which excludes “most epidemiological research from the review.” He further concludes that, “So, the included studies certainly aren’t representative of research as a whole on lockdowns – not even close. Many of the most robust papers on the impact of lockdowns are, by definition, excluded.”
- Even more about the quality. A well-done systematic review discusses why it includes and excludes studies. The authors seem to exclude all of the robust epidemiological studies that showed the positive impact of lockdowns, without a good reason. This just reeks of bias.
All this adds up to a very poor systematic review and meta-analysis, barely meeting the basic standards of such studies. It was highly biased, by excluding studies done by real epidemiologists and public health experts.
As Dr. Meyerowitz-Katz concludes:
All of this adds up to a very weird review paper. The authors exclude many of the most rigorous studies, including those that are the entire basis for their meta-analysis in the first place. … They then take a number of papers, most of which found that restrictive NPIs had a benefit on mortality, and derive some mathematical estimate from the regression coefficients indicating less benefit than the papers suggest.
Fox News, which is not exactly the leader in unbiased reporting, is trying to claim that there is a media blackout against this study. Nothing could be further from the truth. It’s more like there is a scientific blackout against this study, because every single epidemiologist, public health expert, and scientist have been hitting their head against the desk when they read this study.
This is a terrible study with no redeeming qualities. Not that that matters to Fox News.