In an issue of the International Journal of Infectious Diseases, Arshad et. al [1] „Treatment with Hydroxychloroquine, Azithromycin, and Combination in Patients Hospitalized with COVID-19“ calimed that Hydroxychloroquine was associated with reduction in COVID-19 associated mortality. Naturally the journalists jumped on the story withpout proper proofing the data and hypothesis. The conclusion is to pretty for fans of President Trump and Fox News. But is the conclusion right.
Some remarks of the study you can find in another issue Lee, et al [2]
The authors conducted a retrospective cohort study of 2,541 consecutive patients admitted to their health system in Michigan, USA. Patients were separated into four groups: no treatment (n = 409), azithromycin alone (n = 147), hydroxychloroquine alone (n = 1202), and hydroxychloroquine plus azithromycin (n = 783).
The conclusion and relevance of the study
In this multi-hospital assessment, when controlling for COVID-19 risk factors, treatment with hydroxychloroquine alone and in combination with azithromycin was associated with reduction in COVID-19 associated mortality.
is not proven by the data.
Minor remarks on the data
In Table 1 Patient Characteristics by Treatment Group are (at least) two errors.
Age in Years
Age in Years, Mean ± SD, Median (IQR) for HCQ Alone is 53 (64 – 74)The median (Value=53) for „HCQ Alone“ is outside the IQR (64-74). That’s impossible. The mean must be between 64 and 74. Or the correct data is 64 (53 – 74). Wich would better fit with the age in the other groups.
Age n (%)
For the cell „Age / Neither med“ the sum 158 + 251 = 409 is correct, but 38.6% + 64.1 = 102.7%. Replace 64.1 with 61.4 and you get 100%.
Let us assume this are typos and my proposal is correct.
Comparing the treatment groups
On the CDC web site we find the COVID-19 Laboratory-Confirmed Hospitalizations[3].
Because Arshad et. al all don’t provide the age distributen in the same age ranges we compare the groups by the normal distribution and the number of patients older >65YR.
| Age (YR) |
CDC | Total | Neither med | HCQ | AZM | AZM + HCQ |
| 0-4 | 0,5% | 0,0% | 0,0% | 0,0% | 0,0% | 0,0% |
| 5-17 | 0,6% | 0,2% | 0,2% | 0,2% | 0,4% | 0,2% |
| 18-49 |
27,1% | 19,2% | 14,9% | 18,8% | 20,8% | 20,8% |
| 50-64 |
28,9% | 32,5% | 26,9% | 34,3% | 31,5% | 34,5% |
| 65+ |
42,9% | 47,8% | 56,8% | 46,5% | 46,9% | 44,3% |
| Ashar 65+ | – |
49,7% | 61,1% | 48,9% | 46,3% | 45,5% |
The Neither med treatment group is much older than the other groups.
Age 68.1 ± 18.9 // 71 (56 – 83). The average is 5 years and the
median and the upper boundary of IQR is 7 to 9 years higher than in the
other groups. Mean +SD is 87.0 compared to 80.2, 78.8, 80.6; 78.2
At least 25% of the patients in the Neither med treatment group are older than 83 years. The patients with age > 65 years represent 61,4% of the group. In the other groups they represent <50%. This fact alone could account for the higher mortality in this group.
The elderly are at the greatest risk of dying, if infected with this virus. [4]. People above 80 can have a case fatality rate up to 6 times higher than people between 60-69.
As a first guess I assume the null hypothesis
patients <83 years have a mortality of 20%, patients >83 years have a mortality of 50%
Because I don’t have the raw data I can’t prove the hypothesis but the results given by Ashra fit pretty good with this hypothesis. See the example below.
There are some other big difference between the groups. With 60.2%
black patients are over represented in the HCQ Alone treatment group.
Therefore the Neither med treatment group is not comparable with the other
groups.
Examples
We assume that the mortality of people >83 is 50%. It’s possible that
old patients get less attention than younger patients. They have right
to die. This could be a reason for the assumed higher mortality. Because there are less patient of this age in other groups we must exclude them before comparing the groups.
In the Neither med treatment group are 409*25% ~ 102 patients >83, according to the hypotheses 50% ~ 51 die. If we exclude this group we get 307 patients for Neither med / <83 with only 57 (18.6) death. The Neither med / <83 mortality (18.6%) is close to the mortality of the total group (18.1). According to the hypothesis we would assume 112 death (307*0.2+102*0.5). That is close enough to 108. The difference isn’t significant.
Without the raw data we can’t exclude the patients above 83 from the other groups in the same way. We can try it with the normal distribution.If we assume a mortality of 75% for people >83 years we get a mortality
of only 10% in the group „Neither med / < 83yr“. This should be checked
and addressed in the study.
Mortality Age >83 = 50%
| Characteristics | Total | Neither med | HCQ | AZM | AZM + HCQ | |
| N | 2541 | 409 | 1202 | 147 | 783 | |
| Mortality | n | 460 | 108 | 162 | 33 | 157 |
| % | 18,1% | 26,4% | 13,5% | 22,4% | 20,1% | |
| Age | Mean | 63,7 | 68,1 | 63,2 | 63,3 | 62,3 |
| SD | 16,5 | 18,9 | 15,6 | 17,3 | 15,9 | |
| > 83 | % | 12,1% | 25,0% | 10,2% | 12,7% | 9,6% |
| < 83 | n | 2.233 | 307 | 1.079 | 128 | 707 |
| Mortality | <83 | n | 300 | 57 | 101 | 24 | 119 |
| Mortality | <83 | % | 13,4% | 18,5% | 9,3% | 18,4% | 16,9% |
This reduces the mortality of to the Neither med treatment group | Age <83 to the same level as the AZM and AZM+HCQ treatment group.
Let look what happens when the mortality of people older 83 its 100%.
Mortality Age >83 = 100%
| Characteristics | Total | Neither med | HCQ | AZM | AZM + HCQ | |
| N | 2541 | 409 | 1202 | 147 | 783 | |
| Mortality | n | 460 | 108 | 162 | 33 | 157 |
| % | 18,1% | 26,4% | 13,5% | 22,4% | 20,1% | |
| Age | Mean | 63,7 | 68,1 | 63,2 | 63,3 | 62,3 |
| SD | 16,5 | 18,9 | 15,6 | 17,3 | 15,9 | |
| > 83 | % | 12,1% | 25,0% | 10,2% | 12,7% | 9,6% |
| < 83 | n | 2.233 | 307 | 1.079 | 128 | 707 |
| Mortality | <83 | n | 141 | 6 | 39 | 14 | 81 |
| Mortality | <83 | % | 6,3% | 1,9% | 3,6% | 11,1% | 11,5% |
OMG not treatment is better than any treatment.
Lets hav a closer look to the composition of the HCQ alone treatment group.
Is there a difference between the groups that could be hold accountable for the low mortality?
Lets look at the race. The ration between black and white in the HCQ alone treatment group is 2:1. In the Neither med treatment group the ratio is 1:1. If the mortality of black patients is only 10% this would fit with the results.
BTW: Why is the ratio black:white in the HCQ alone group so high and in
the „Neither med“ so low? Total group 5:3.
My first guess for a null hypothesis would be
mortality of non black is 25%, black 10% and >83 75%. (And AZM may raises the mortality. HCQ has no effect.).
I guess this hypothesis satisfies the mortality in Table 1 for the total group, the „Neither med“ and the „HCQ alone“ group pretty good. The rest explains the use of AZM.
Conclusion
The treatment groups are not representative against each other. The data has to be evaluated against some more complex hypotheses and a deeper analysis of the treatment groups is required. (But: Correlation is not causality.)
The simple assumption that mortality depends on age and race only can
explain some results. The results should be evaluated against ranges of
age 18-29,30-39,….,90-99.
„The result that … treatment with hydroxychloroquine alone and
hydroxychloroquine + azithromycin was associated with a significant
reduction in mortality … “ is not demonstrated as a simple examples –
without access to raw data – demonstrate.
- [1]Samia Arshad, Paul Kilgore, Zohra S. Chaudhry, Gordon Jacobsen, Dee Dee Wang, Kylie Huitsing, Indira Brar, George J. Alangaden ,Mayur S. Ramesh, John E. McKinnon, William O’Neill, Marcus Zervos, Henry Ford COVID-19: Task ForceTreatment with Hydroxychloroquine, Azithromycin, and Combination in Patients Hospitalized with COVID-19
Open AccessPublished: July 01, 2020 DOI: https://doi.org/10.1016/j.ijid.2020.06.099↩ - [2]Todd C. Lee, Lauren J MacKenzie, Emily G. McDonald, Steven Y.C. Tong: An Observational Cohort Study of Hydroxychloroquine and Azithromycin for COVID-19: (Can’t Get No) Satisfaction
Open AccessPublished: July 02, 2020 DOI: https://doi.org/10.1016/j.ijid.2020.06.095↩ - [3]COVID-19 Laboratory-Confirmed Hospitalizations – Preliminary data as of Jun 27, 2020 accessed 09 Jun 20<7a>↩
- [4]Max Roser, Hannah Ritchie, Esteban Ortiz-Ospina and Joe Hasell (2020) – „Coronavirus Pandemic (COVID-19)“. Published online at OurWorldInData.org. Retrieved from: ‚https://ourworldindata.org/coronavirus‘ Accessed 07 Jul 2020↩
