2/7/21 Note: I had someone claim that I was being selective by only showing the first set of studies. I honestly hadn’t realized that I missed some since I was writing late. This will be completed, but I have added the second group image with the same criteria used. That one only has one study worth digging into. I’ll do the same with the third group when I have time.
Ivermectin continues to be pushed for COVID when there isn’t clear evidence to indicate that it provides any benefit. Hill et al (2022) indicate that “the results suggest that the significant effect of ivermectin on survival was dependent on largely poor-quality studies.” They also go on to describe a number of fraudulent studies that have greatly influenced the outcomes of some meta analyses and conclude “These instances suggest that the data available to the support the use of ivermectin for COVID-19 are not reliable” and have illustrated how much difference removing completely fraudulent to those with high risk or some concerns change the outcome considerably. They also suggest that publication bias (see Publication Bias: A Brief Review for Clinicians) is also overestimating the benefit of ivermectin.
There is a website that people often use to try to claim that it works, but clearly they don’t understand statistics. It uses 77 studies as of this writing to try to promote ivermectin.
This is a screenshot of that website with the studies highlighted with different colors. Someone without much background in science or statistics might look at the column on the right and quickly conclude that these studies indicate how well ivermectin works. In fact, that graph is intended to completely misrepresent the data.
The third column in the table has a pair of numbers in [brackets]. They indicate a range from low to high known as the 95% confidence interval, often referred to as the 95% CI, or in this table the [CI]. Without going into a deep discussion on what these values mean, there is only one important fact about the studies shown with the red highlight – they include the value of 1.00 in that range. When that is the case, it indicates that there is no statistical difference between the study group and the control group, or to put it in simple English, there is no evidence that the treatment worked any better than not using the treatment. Without even opening those individual studies, one can see that over half don’t show evidence for efficacy.
The other important concept that can rule out studies in this table is the sample size. It’s simply the number of participants in each arm of the individual studies. This gets quite a bit more technical but it is related to another concept known as statistical power, which is a measure of how effective a study is at determining differences between the two arms of the study. This is a number that can be calculated, but with experience, it becomes relatively easy to quickly identify those that have too small of a sample to really provide any meaningful conclusions. A slightly more complicated but still readable discussion of this can be found here.
As a quick example, assume you have an opaque jar with 100 different marbles, 25 of each color – red, blue, green, and yellow. If one doesn’t have knowledge of the composition of colors in the jar and was asked to provide an estimate of the proportion of each, obviously they will get a much closer estimate of those percentages by taking 50 marbles out instead of 5. That’s a very basic way to understand a little bit of the relationship between those two concepts.
This is an analogy of the above table. The ones that remained after eliminating the red ones are highlighted yellow because they didn’t sample enough marbles to prove their hypothesis.
That leaves only FOUR of the initial 77 studies to assess for validity (highlighted green in the table).
Borody
This study doesn’t appear in a peer reviewed journal nor could I find it on a preprint server. Typically I would look at the date it went on a preprint server as the next step. If it has been a long time, that generally means that no reputable journals have accepted it. The fact that it’s not in either location is a MAJOR red flag. A little further digging explained why.
de Jesús Ascencio-Montiel
Guess who is behind the website where the screenshot came from. Yes, de Jesús Ascencio-Montiel. Given that he is clearly misrepresenting data, it puts into anything he is behind into question. In fairness though, this one has been accepted for publication in what at first glance looks to be a legitimate journal from Elsevier, the Archives of Medical Research. One of the easiest ways to determine if a journal is legitimate is to assess if it uses a pay-to-publish model. If it is, it can be discarded.
The Archives of Medical Research is a tricky journal in this regard. It uses both the traditional publishing model and the pay-to-publish model, which means one must assess each individual article. It’s pretty easy with his. Open access means pay-to-publish. Look at the bottom right of the image.
de Jesús Ascencio-Montiel can be discarded.
Mayer
Mayer doesn’t show up in preprint or published locations either. Discard it. It’s garbage.
Merino
The title of this should be a dead giveaway that it’s not a useful analysis. “Ivermectin and the Odds of Hospitalization Due to COVID-19: Evidence from a Quasi-experimental Analysis Based on a Public Intervention in Mexico City.”
Once again, it’s another that doesn’t appear to have been published and can only be found on a preprint server. This is a good example of why it’s helpful to look at the posting date on the server. It was placed there on May 4, 2021. Given how long ago that was, it’s a very safe bet that this will never be accepted for publication either.
Soto-Becerra
This is another preprint that has not been published or peer reviewed in over four months. It was posted on October 8, 2020.
Summary
With some very basic knowledge, one can easily rule out the validity of studies. Out of these 77, not even one needed to be evaluated from a research methodology and science standpoint by reading it, which is considerably more challenging.
Stay away from ivermectin as a treatment for COVID or anyone who is pushing it. They are either scientifically illiterate, acting from a basis of politics and not science, or are financially profiting in some way.
