Palladium has a long history of statistical modelling, from public health to zoonotic disease and even agriculture. As modelling takes centre stage in the fight against COVID-19, Palladium Senior Director Scott Moreland reflects on the needs and limitations.

Modelling is receiving more than the usual attention lately, with commentary on the limitations of models by the White House COVID-19 task force, the Washington Post, and others.

Some jurisdictions, such as the province of Ontario in Canada, have recently released extensive COVID-19 models to the public in an effort to underline the urgent need for social distancing. Other leaders and public health agencies have chosen to share varying levels of detail. But how should we be thinking about the validity of these models, and to what extent should they drive policy and behaviour?

As we continue with our own responses to the COVID-19 pandemic, it’s useful to recall that models fall into three (inevitably overlapping) categories, and how these can best be applied:

1. Research models are intended to establish the empirical basis for linkages between complex factors (i.e., will X influence Y?).

2. Policy models take research model results (when they exist) and can serve as tools to support decision making (i.e., should we change X to change Y?). While any model produces outputs with apparent numerical precision, the emphasis with policy models is qualitative, not quantitative.

3. Predictive or planning models focus on what the size of changes will be (i.e., how much will Y go up if I change X by a certain amount? Will it rain today?).

For the current COVID-19 modelling, most of the virus models (such as those coming out of University College London or the University of Washington) are based on classic disease transmission models that have been around for many years. The architecture is well established.

The challenge is in filling in the parameters upon which the models turn; we lack a strong research base for the current pandemic to set many of these parameters. For example, if X% of the population practices social distancing, what will be the impact on incidence? What is the impact of co-morbidity on the rate of amplification? The answers to such questions require impact evaluation studies that can take time and money to answer, neither of which are available in the current crisis.

Needs and Limitations for Policy

We simply don’t have all the answers, and yet important policy decisions are needed. Does this mean we shouldn’t use models? On the contrary, models are still our best approach, if they are seen and used within their limitations.

In the early days of HIV/AIDS, my team worked in Nigeria with the Federal Ministry of Health as the country and world began to face the epidemic. We used a very simple epidemic projection model that fit a curve statistically to the number of observed cases and projected the numbers forward. When the results were presented to then President Obasanjo, he was so impressed that he created a new multi-sector agency to coordinate the county’s response.

From a “scientific” point of view, the model behind the projections was based on minimal data (three observations at most!) and the results likely would not have passed muster in a peer-reviewed journal. Yet, few would argue that the President’s decision was wrong.

Similar decisions are now being made by governments across the globe. Should we wait until we have all the data we need, and all the relationships modelled? In any other situation, yes, life-and-death decisions should be based on evidence. But this crisis requires agility and the quickest possible response. As data scientists, we need to keep our methods as simple as possible and focus squarely on the highest-priority outputs.

This isn’t the first pandemic we’ve faced, and with rigour and planning, we’ll be even better equipped to model the next one.

Scott Moreland is Senior Director of Palladium's Data, Informatics & Analytical Solutions practice. Contact info@thepalladiumgroup.com to learn more.