Why the Operation Warp Speed Vaccine Studies Aren’t Limited to Severe Disease

[Editor’s Note: a version of this article was first published on Nov. 13 on the Johns Hopkins University Coronavirus Resource Center. —LT ]

The COVID-19 Operation Warp Speed (OWS) trials have taken some criticism in the medical press, and lay press, for evaluating what some consider to be “trivial” characteristics of mild COVID-19 disease.

Some are arguing that it would make more sense to evaluate vaccine efficacy entirely on ability to prevent severe COVID-19 disease, hospitalizations and deaths.

There’s a reason why the trials weren’t focused entirely on those narrow parameters. Allow me a few minutes to explain.

The trial protocols for four major vaccine developers – Pfizer, Moderna, AstraZeneca and Janssen (Johnson & Johnson) — show that the primary endpoint of the OWS vaccine trials is a reduction of coronavirus disease.

The disease is defined broadly in this case, and includes reducing the signs and symptoms of mild COVID-19 illness (e.g., headache, cough, fever, myalgias, signs of loss of taste and smell).

By including mild COVID-19 disease within a primary endpoint, the trials can capture the wide range of symptoms, including sometimes surprising symptoms, observed in patients with COVID-19.

This approach will ultimately allow for vaccines, if successful, to be given to the widest group of people possible – not just a couple important subpopulations. The trials should provide the kind of evidence necessary for efficient real-world rapid assessment of the vaccines’ effect on COVID-19.

For companies developing vaccines, inclusion of mild COVID-19 disease within a primary endpoint for the OWS trials makes sense; these companies recognize the wide range of COVID-19 disease, from mild to severe, and want to produce an effective vaccine with widespread benefit.

Reading further into the vaccine trial protocols shows that a wealth of information is being collected for analysis. Within each trial, there is a deeper evaluation of disease severity: the trials have an endpoint that includes measurement of disease severity at diagnosis and over three weeks of follow-up for every participant who develops COVID-19.

If one looks in the tables at the end of the protocols, (Table 17 in the Moderna protocol, Table 4 in the AstraZeneca protocol, and Appendices 2, 6-8 in the Janssen protocol), one sees that every person in the trials with COVID-19 infection is intensively followed from the time of diagnosis over the next three weeks with standardized assessments of their signs and symptoms. The assessment includes hard objective measurements, such as the use of medical facilities, hospitalizations, and what as investigators we all hope is rare – death.

At the time the vaccine trials were designed in the spring and summer, there was no consensus about the frequency with which those who volunteered to enroll would actually get COVID-19. More importantly, it was unknown what the spectrum of COVID-19 disease among trial participants would be.

In fact, we were pretty certain we would not see what, at that point, had been the norm in the pandemic (i.e., large numbers of people presenting to hospital emergency rooms with full-blown end-stage pneumonia). We suspected that as testing became more widely available in the summer and fall, the vaccine trials would begin to reflect some of this change. We expected participants would report to their investigators earlier, with milder COVID-19 symptoms than what was, at that time, being seen in medical practice or in any published literature.

Healthy volunteers who enroll in clinical trials, especially ones with the complexity and follow-up required in the COVID-19 vaccine efficacy trials, generally have to be interested and concerned about the disease.

The design of the OWS trials involves active disease surveillance. There is essentially no barrier to COVID-19 testing for the 30,000 participants per study. Moreover, these 30,000 participants are knowledgeable about COVID-19, likely anxious about getting the disease, and the experience of serving as a trial participant makes them attuned to any aspect of the disease, all the time. Participants in the trials receive weekly calls/texts about coming to the clinic at the first signs of any respiratory illness or fever. Importantly, trial participants have essentially unlimited access to COVID-19 testing.

Thus, we felt that a clinical classification of COVID-19 disease severity at the time of diagnosis would not really provide an accurate reflection of a participant’s COVID-19 illness. We expected that most trial participants would get worse after an initial diagnosis. As such, the real endpoint evaluation of disease severity – and hence the ultimate “primary endpoint” – would be the constellation of signs and symptoms that were collected over the two to three weeks post-diagnosis. The section outlining the detailed follow-up of COVID-19 cases within each study had its own schematic diagrams/tables and was “costed out” for site funding separately.

One of the harmonizing aspects across all the vaccine trials is that participants with COVID-19 are queried daily with a standardized questionnaire evaluating their signs and symptoms on a mild-to-moderate-to-severe scale. Samples for viral shedding in saliva or the nose are taken every second to fourth day, depending on the protocol, to evaluate the duration for which the virus is shed. Duration of fever and pulse oximetry data are also recorded.

These data allow one to grade the rate of progression, duration of symptoms, duration of signs, duration and severity of systemic effects, and need for any follow-up medical interventions (e.g., physician visits, telemedicine visits, ER visits, hospitalizations, and any complications of hospitalizations). Evaluations of these data are blinded and will be analyzed between the vaccinated and placebo recipients to determine whether disease duration and severity are reduced.

Data outlined in the prior paragraph are listed as secondary endpoints in the trials. While “relegated” to this section, it does not mean their importance is secondary. This was highlighted in a recent paper on using burden of disease as the primary endpoint in COVID-19 vaccine studies by Mehrotra and colleagues, as well as recent analyses of data from the monoclonal antibody studies conducted by Lilly and Regeneron.

The monoclonal antibody studies do a nice job of aggregating medically complicated disease to show the benefits of early use of the antibodies. They also show one approach to compiling recorded signs and symptoms of COVID-19 to show clinical benefit to patients over time. This offers a model to structure analyses both within and between individual and collated vaccine studies. In addition, the prospective close follow-up that will occur in the vaccine studies lays the groundwork for future studies and maximizes what we learn from the 150 expected COVID-19 cases in each vaccine trial of 30,000 participants.

It is unlikely that, in any individual trial, we will see enough hospitalization and death to evaluate efficacy of a vaccine versus placebo for severe COVID-19 disease. Obtaining this information within a trial would require lengthening the time to determine whether a vaccine works by 18 months.

That is simply too long to wait in a fast-moving pandemic.

Vaccines for other diseases offer a useful perspective about the inclusion of mild illness as a primary endpoint within a trial: data from other vaccines show that those capable of modifying outpatient/modest illnesses have almost universally been capable of modifying more severe illness. With most vaccines, efficacy in reducing the severe spectrum of a disease is actually somewhat higher than for mild aspects of a disease.

Let us hope that vaccination will easily produce discernible, measurable, medically important differences in COVID-19 disease in the populations enrolled in the trials. The current surge in activity in the U.S. reminds us of the urgent need for vaccines.

Dr. Larry Corey is an internationally renowned expert in virology, immunology, and vaccine development and a leader of the COVID-19 Prevention Network (CoVPN), which was formed by the National Institute of Allergy and Infectious Diseases at the U.S. National Institutes of Health to respond to the global pandemic. He is a Professor of Medicine and Virology at University of Washington and a Professor in the Vaccine and Infectious Disease Division and past President and Director of Fred Hutchinson Cancer Research Center.

Sources:

¹ Pfizer. PF-07302048 (BNT162 RNA-Based COVID-19 Vaccines) Protocol C4591001. 2020. https://pfe-pfizercom-d8-prod.s3.amazonaws.com/2020-09/C4591001_Clinical_Protocol.pdf.

² Moderna TX. Protocol mRNA-1273-P301, Amendment 3. 2020. https://www.modernatx.com/sites/default/files/mRNA-1273-P301-Protocol.pdf.

³ AstraZeneca. Clinical Study Protocol – Amendment 2 AZD1222- D8110C00001. 2020. https://s3.amazonaws.com/ctr-med-7111/D8110C00001/52bec400-80f6-4c1b-8791-0483923d0867/c8070a4e-6a9d-46f9-8c32-cece903592b9/D8110C00001_CSP-v2.pdf.

⁴ Janssen Vaccines & Prevention BV. VAC31518 (JNJ-78436735) clinical protocol VAC31518COV3001 amendment 1. 2020. https://www.jnj.com/coronavirus/covid-19-phase-3-study-clinical-protocol.

⁵ Moderna TX. Protocol mRNA-1273-P301, Amendment 3. 2020. https://www.modernatx.com/sites/default/files/mRNA-1273-P301-Protocol.pdf.

⁶ AstraZeneca. Clinical Study Protocol – Amendment 2 AZD1222- D8110C00001. 2020. https://s3.amazonaws.com/ctr-med-7111/D8110C00001/52bec400-80f6-4c1b-8791-0483923d0867/c8070a4e-6a9d-46f9-8c32-cece903592b9/D8110C00001_CSP-v2.pdf.

⁷ Janssen Vaccines & Prevention BV. VAC31518 (JNJ-78436735) clinical protocol VAC31518COV3001 amendment 1. 2020. https://www.jnj.com/coronavirus/covid-19-phase-3-study-clinical-protocol.

⁸ Mehrotra, DV, et al. “Clinical Endpoints for Evaluating Efficacy in COVID-19 Vaccine Trials.” Annals of Internal Medicine. https://doi.org/10.7326/M20-6169. Published: 22 October 2020(https://doi.org/10.7326/M20-6169. Published: 22 October 2020).

⁹ Chen, Peter, et al. “SARS-CoV-2 Neutralizing Antibody LY-CoV555 in Outpatients with Covid-19.” The New England Journal of Medicine. https://www.nejm.org/doi/full/10.1056/NEJMoa2029849. Published: 28 October 2020(https://www.nejm.org/doi/full/10.1056/NEJMoa2029849. Published: 28 October 2020).