Digital Tools Helped Enable COVID-19 Vaccine Trials. What are the Lessons Learned?

Digital tools played a critical role in accelerating the development and evaluation of COVID-19 vaccines, according to leaders at the companies driving this effort. 

Speaking at a recent virtual panel discussion organized by the Galien Foundation (and available here), leaders from Pfizer, AstraZeneca, Janssen, Moderna, and the CRO IQVIA shared their experiences leveraging digital tools for vaccine development.

Data Deluge

The first striking observation was simply the staggering scale and raw intensity of the undertaking.  Pfizer’s phase 2/3 study, for example, involved around 44,000 subjects, 156 research sites in six countries – and was executed in four months. Janssen shared similar data – a 45,000 person study, which managed to recruited 3,000 subjects on a single day.

These are eye-popping figures.

The volume and velocity of trial data – far outside the norm for conventional clinical studies – presented urgent challenges to vaccine developers, who responded by leaning into existing data management platforms, and supplementing them with fit-for-purpose tools such as highly customized dashboards.  This enabled study sponsors to monitor a range of vital study parameters — including site start-up, participant diversity, data entry, and data cleaning — and where necessary, make adjustments on the fly. 

A particular challenge for COVID-19 studies was anticipating where the virus was likely to be most prevalent – regions where the impact of the candidate vaccine would be most readily ascertained.  All the companies claimed to use a range of analytic techniques and diverse data sources (such as mobility data) to identify the most promising study locations.

The “Last Mile Touch”

Another important challenge highlighted by several speakers involved the implementation of technology, the “learning curve that we had to bring our sites and investigators and staff on board,” as one leader put it. 

Some companies took to deploying “virtual study coordinators” who were “trained up on the tech” and could “bring folks up to speed with the new mobile devices, and help with the back office.” Help desks were staffed up to manage the increased call volume, and tech coordinators were also assigned to research sites. 

Takeway: Technology may be powerful, but people are required to make it effective. 

As one speaker said:

“Technology works, but actually technology had to be complemented with humans….you do need that last mile touch…one of the things we realized in the scale up is how do you bring the human and technologies together, complementing each other so that we can make sure this delivers the promise, what it’s designed for.”

Moreover, study sponsors discovered that technology wasn’t equally embraced by all participants, “some of whom will find it easier to use a tool than others.”  

Consider the question of using an electronic diary versus a paper diary.

“Sure,” a speaker suggested, “the electronic is easier for us, for many people around the world, actually putting X’s on a piece of paper is easier than finding the app on their iPhone.” 

Another speaker noted many participants “would not necessarily respond to super techie reminders,” and thus sponsors recognized the need to add “at times the human touch to really complement the technology when we need it.”

The New Normal?

The speed of COVID-19 vaccine development inevitably raises the question of whether the approaches used to accelerate the process will become standard, and represent the new normal. 

On the one hand, there was clearly the ambition to leverage the tools and learnings from the vaccine experience, as suggested by one company’s mantra, “no going back.” 

At the same time, speakers acknowledged just how exceptional these circumstances truly were – from the worldwide prevalence of disease to continuous (versus episodic) engagement with regulators, cited by all sponsors as critical to study acceleration. COVID-19 vaccine studies also operated under economics, and in the presence of resources that were far from typical.

The toll on all the people involved in executing these studies was also palpable – speakers described teams working “flat out” for months, to the point of utter exhaustion. It felt like the sort of superhuman effort that could be mustered and sustained only for a cause as cataclysmic as a global pandemic.

The ability of data platforms to support (apparently effectively) large volume COVID vaccine studies may also not speak to their ability to support the more usual trials companies sponsor, where the numbers are significantly lower, cost is a more pressing factor, and identifying and recruiting difficult-to-find subjects a recurring challenge. It will also be important to pressure-check the triumphalist story shared by these digital champions and stakeholders against reality, to the extent such an independent after-action report is possible.

Fix vs Reinvent?

Perhaps the most intriguing observation emerging from the discussion was a point made by several speakers, and elegantly summarized and extended upon by the moderator, Jessica Federer (former Chief Digital Officer of Bayer, and currently Managing Director of Huma Health):

“Something that we hear from so many pharmaceutical companies and manufacturers [is] that we’re using new technology, but we’re retrofitting our old processes and we’re trying to do the same things we’ve always done using new tech. And that clash is creating a complexity and it’s not shortening the timelines and it’s causing barriers. And the challenge for us as an industry is to  try to do new things with new tech instead of the same old ways we’ve always done it.”

This perfectly captures the challenge of technology implementation in medicine, as I’ve discussed here.  One prominent example: electronic medical records, which hew closely to traditional physician notes, in a deliberate effort to ease adoption. 

The “fix vs reinvent” tension turns out to pervade the history of technological innovation (see here), including the classic example of electrification of factories. Data suggest that electrifying legacy factories – historically built in a dense three-dimensional configuration to leverage the single source of steam power – by swapping in an electrical generator didn’t do much to goose productivity. It wasn’t until innovators completely rethought the design of the factory and developed an entirely new layout and workflow that huge productivity gains were finally achieved.

In the case of factories, innovation was spurred by the economic gains associated with improved efficiency. In medicine, the economic incentives, as Todd Park recently discussed, can be far less intuitive – perhaps the most substantial barrier faced by aspiring healthcare innovators. (Harvard’s Zak Kohane makes a similar point here).

Bottom Line

Vaccine developers leveraged digital platforms to manage the staggering data requirements and clinical trial needs. Successful implementation of new technologies required people to assist with the last mile. The exceptional global circumstances associated with vaccine development suggest we ought to exercise caution before generalizing from the experience, even as we’re inspired by the results. Coming up with novel ways to leverage technology for evidence generation, rather than adapting new technologies to established procedures, represents a future frontier; aligning economic incentives will be critical.