Why pandemic projections are bewilderingly different

If you listen to the widely varying projections about how this COVID-19 pandemic will play out, and you find yourself sliding into a combination of intellectual stupor and disbelief, you are not alone.

Yes, we all recognize that the people and institutions who are developing these models are experts in epidemiology, public health and data analytics. But when one hears a forecast of somewhere between 60,000 and 240,000 deaths, it's hard not to wonder whether maybe these people are working with a dartboard, not a supercomputer.

If a contractor gave you a bid for your kitchen remodel of somewhere between $60,000 and $240,000, you'd be calling another contractor before the first one drove away.

Why is modeling a viral pandemic so hard? It's because of two things: 1) exponential growth bias and 2) what happens to exponential growth models that have to depend on limited and uncertain data.

Acknowledging that individuals have different aptitudes for statistics, it turns out that many of us share a cognitive "blind spot" in our understanding of the difference between linear growth and exponential growth. We understand linear growth pretty well, where a number increases by a fixed amount repeatedly over time (2, 4, 6, 8, 10, 12 … ). But we have a much harder time comprehending exponential growth, where a number doubles repeatedly over time (2, 4, 8, 16, 32, 64 … ).

This idea of "exponential growth bias" appears more often in personal finance literature than it does in public health conversations. Here's a common, finance-based, exponential growth bias quiz:

If I give you $1 and it doubles every day, at the end of a 31-day month, how much money will you have?

The quiz demonstrates the power of compounding interest, wherein a small sum can grow exponentially over time. Because compounding interest works the same way whether one is saving money or borrowing it, failing to appreciate its power often leads consumers to either save too little or borrow too much. They think they can make up for lost time, but it's hard to catch up with exponential growth.

The COVID-19 virus infection has grown exponentially, and its spread is typically reported by its "doubling time" — how many days does it take for the number of cases (or deaths) to double (ourworldindata.org/coronavirus). Early on in a pandemic, the virus is moving quickly and the number of new cases can double in a few days. As the spread slows, doubling times increase. As the infection stabilizes, the exponential growth turns to linear (the same number of new cases develop each day) and finally the curve drops as the pandemic fizzles.

Here's the key concept: Exponential growth means that very small changes at the beginning can lead to very large changes over time.

I don't golf very often because I have an "exponential growth" kind of swing. My shots start out looking linear, straight down the fairway, but then begin careening harder and harder to the left. They call it a "hook" for a reason, which is why exponential graphs arc like a snowboard pipe, whereas linear graphs look like a ramp.

This, then, is the Achilles' heel of exponential growth modeling: Small changes in the data used to run the model can lead to enormously different future projections, and the further one tries to peer into the future, the more magnified those differences can be.

On the golf course, a 150-yard hook might be just 30 yards off the fairway, but a 200-yard hook might make it all the way into the next county. The more airtime it has to hook, the further off course it can travel.

We've heard from the earliest days of this pandemic about the lack of COVID-19 testing. It's easy to understand how important it can be to an individual patient, but from a public health standpoint, the lack of testing means we are "flying blind" with no way to accurately track the growth and spread of the virus, a data deficiency exacerbated by the fact that a significant number of infected persons have few if any symptoms. Without broad-based, communitywide testing, it's difficult to get accurate numbers to put into any of the COVID-19 projection models.

In Minnesota, the newest projections by the governor's team of experts predict a peak of 3,000-5,000 COVID-19 patients requiring ICU hospital care sometime between mid-May and mid-July. The governor explained that the new model took into account a revised R0 — that's the number of people that one infected individual subsequently infects. The R0 went from 2.4 to 4.0, a big jump that will only be further amplified in an exponential growth model.

The principle that "small changes early can make huge differences later" is exactly why social distancing done early — when the interest, so to speak, has time to compound — can have powerful results a few weeks down the line. One fewer infection now prevents that person from spreading it to four others, which would have become 16, which would have become …

One might sincerely ask, "If modeling is so imprecise, why bother?" The answer lies within a quote from statistician George Box, who wrote: "All models are wrong, but some are useful."

A widely sourced (and generally more optimistic) COVID-19 projection website is from the University of Washington's Institute for Health Metrics and Evaluation. The site's mortality graphs contain a solid line of where COVID-19 has been, and a dotted line for where they think it is going. The dotted line into the future is surrounded by "a shaded area [that] indicates uncertainty." It's their way of saying, "Our best guess is that it will follow this dotted line, but it could fall anywhere in this range."

Right now there is entirely too much shaded area. But until we acquire more robust data, we're all going to have to put up with a certain amount of uncertainty.