COVID-19 fight has relied too much on uncertain math modelling, some scientists say

The inadvertent side effect may be public distrust of government officials fighting the coronavirus, critics argue

Article content

The first several pages of the Public Health Agency’s epidemiological update last week struck an optimistic note.

The number of COVID-19 cases, hospitalizations and deaths were all falling steadily after a major early winter surge.

Then came slide 13 in the presentation, an alarming new model of what could happen next.

Based on more transmissible variants of the virus spreading widely, it showed an almost vertical spike in the case count should public health restrictions be eased. An only slightly less mountainous climb would occur into March and April, even if current lockdown measures were kept in place.

The only way to avoid a feared third wave of COVID-19 illness, the report suggested, was by imposing new restrictions.

It was a sobering message, but hardly unfamiliar by now. Almost since the pandemic began, scientists have been using high-level mathematics and powerful computers to churn out models of how the virus might spread and affect Canadian society, and what mitigation measures would do to slow it down.

Advertisement

Story continues below
This advertisement has not loaded yet, but your article continues below.

Article content

More On This Topic

  1. Deputy Chief Public Health Officer Dr. Howard Njoo:

    Canada's COVID models have been largely accurate, but worst cases have not materialized

  2. People wear face masks as they walk along a street in Montreal, Sunday, January 31, 2021.

    Increased cases of COVID variants spell potential third wave, Tam warns

Few experts deny that modeling has been of at least some value, while a National Post analysis found that most federal models have accurately predicted case numbers. Measures taken in response seem to have staved off nightmare scenarios of sickness and death.

But a year into the pandemic, skeptics worry that the battle against COVID-19 has relied too heavily on mathematical projections that can be undermined by sparse data and an unpredictable virus — and are often released with scant mention of their limitations.

The inadvertent side effect may be public distrust of government officials fighting the coronavirus, critics argue.

“When you put out a model that is not credible and it turns out to be ridiculed, you’ve threatened the integrity of the whole health-care system,” argues Ed Mills, a Vancouver-based epidemiologist and part-time McMaster University professor. “So don’t be surprised if people start not believing what the public health agencies say.”

In fact, there’s already skepticism about those federal predictions of a third wave in March fuelled chiefly by the B.1.1.7 mutation of the SARS-CoV-2 virus — the so-called U.K. variant.

Ontario released a similar model on Feb. 11, and a recent decline in cases seems to be flattening off. But Mills, an adviser to the Gates Foundation, and others argue there is simply not enough evidence to conclude the trend is all but inevitable.

Advertisement

Story continues below
This advertisement has not loaded yet, but your article continues below.

Article content

Projections for Canada are based on what happened in the U.K. in late December, when the more-transmissible variant contributed to a troubling surge in cases, noted Dr. Zain Chagla, an infectious disease specialist at McMaster University.

But it’s unclear, he said, how much of that spike can be blamed on the variant, and how much on the somewhat eased restrictions in place in England at the time, plus the inevitable rise in contacts around Christmas.

Don’t be surprised if people start not believing what the public health agencies say

In Denmark, meanwhile, the U.K. variant is prevalent, yet the case count has fallen for weeks, albeit in the wake of a strict lockdown, noted Chagla.

“In terms of predictions for a third wave, I just don’t know,” said epidemiologist Prabhat Jha, director of the Toronto-based Centre for Global Health Research. “We don’t have enough data.”

Despite such reservations, though, modeling has many boosters in the scientific community.

While no one should view the science as some kind of crystal ball to peer into the future, it has played a “huge role” in pandemic planning , says one prominent expert in the field.

“I think it’s been remarkably successful,” said Kumar Murty, a University of Toronto mathematics professor and chair of Ontario’s science modeling table. “(But) models are not a panacea or magic or anything. They are a tool. They are one tool to help understand a complex phenomenon.”

It’s easy to complain about a model being wrong when the actual outcome differs from the projection, he said. But such criticism ignores the changes in policy and personal behaviour that followed the model, often in reaction to it, said Murty.

Advertisement

Story continues below
This advertisement has not loaded yet, but your article continues below.

Article content

“We never wanted to be left with the scenario that it did happen and we weren’t prepared,” says Chagla.

To try to foresee what might happen — and guide preparations — modelers crunch real-world data through mathematical equations to approximate the future course of a disease or the impact of interventions.

The field has been around literally for centuries, the first recorded model predicting in 1760 that universal smallpox inoculation would boost life expectancy by more than three years.

The math application has boomed in recent months as scientists worldwide have put aside their regular pursuits to address the COVID-19 crisis. The medrxiv.org preprint site and the PubMed registry list between them about 1,300 studies that mention modeling and COVID-19 in the title or abstract.

But there have been questions since the beginning about some of the results.

Models are not a panacea or magic or anything. They are a tool

U.K.’s Imperial College jolted the world with modeling that indicated the U.S. would see 2.2 million deaths if it did not take action to limit COVID-19’s spread.

In Canada, early models foresaw an almost apocalyptic impact on health care.

The University of Toronto-led COVID-19 Modeling Collaborative suggested last March that under a “conservative” scenario as many as 5,000 coronavirus patients at a time would need to be treated in an intensive-care unit by mid-May, many requiring ventilators that were in limited supply.

To make room for that kind of deluge, hundreds of thousands of elective surgeries were canceled across Canada, as governments desperately sought out new supplies of breathing machines.

Advertisement

Story continues below
This advertisement has not loaded yet, but your article continues below.

Article content

COVID-related hospitalizations didn’t peak in Ontario until last month during the pandemic’s second wave, when ICUs cared for up to about 400 people daily. It was a serious strain on the system and weary health professionals, but less than a tenth of what that initial model had predicted.

Modeling early in the pandemic also suggested schools could be a major vector for the virus, leading to widespread closures in the spring.

More recent projections have reached a different conclusion, though the role of schools as a possible accelerator of the pandemic continues to be hotly debated.

Beyond this continent, models by Imperial College predicted a huge death rate in south Asia, more than 2.5 million fatalities in Pakistan alone by last November, noted Dr. Zulfiqar Bhutta, co-director of the Centre for Global Child Health at Toronto’s SickKids Hospital.

Others projected a smaller, but also hugely inaccurate toll, he noted.

Pakistan has had about 12,000 deaths and, as it turns out, politicians largely ignored the foreign modeling, Bhutta said.

Perhaps in response to such prognostications, though, India imposed lockdowns last spring that caused widespread hunger among migrant workers. Its COVID death rate has also remained low, one of the mysteries of the pandemic.

“The kind of lockdowns that we have seen here in the West are just an impossibility in many geographies … they would lead to mass starvation,” said Bhutta. “These models have real consequences if they are applied haphazardly.”

Advertisement

Story continues below
This advertisement has not loaded yet, but your article continues below.

Article content

Mills says models produce by Seattle’s Institute of Health Metrics have become the world’s most reliable. But the University of Washington institute generated controversy itself when it predicted in April that the U.S. would see only about 60,000 total deaths, a model President Donald Trump seized on as he tried to downplay the pandemic’s gravity.

What, if anything, went wrong?

Especially early on, models likely erred by treating populations as homogenous, though some people were more or less resistant to the SARS-CoV-2 virus because of pre-existing immunity or other reasons, said Jha.

Mills argues that mathematicians often have failed to validate their models by running past data through them to see if they could accurately predict what has already happened. Or such data was simply unavailable.

And much is unusual or unknown about the virus, he said.

Murty says modelers have, in fact, worked hard to incorporate new evidence about the disease into their work.

Ontario’s Fields Institute for Research in Mathematical Sciences, which he directs, even conducted a survey to determine what percentage of people followed public health edicts on the coronavirus. The finding was about 70 per cent, information that can now be input into models on the potential impact of anti-COVID measures, said Murty.

Still, Jha worries that governments have placed too much emphasis on modeling and not enough on empirical study of the pandemic as it unfolds. He said he and academic colleagues, for instance, had to take the initiative to conduct a large “sero-prevalence” study to examine what percentage of Canadians have antibodies to the new virus — hard evidence of how widely it has spread.

Advertisement

Story continues below
This advertisement has not loaded yet, but your article continues below.

Article content

“Every past public health pandemic that has been successfully dealt with has also created new knowledge,” said Jha. “Aside from the extraordinary success in vaccines, this pandemic has not produced extensive new knowledge.”

If there is one thing that critics and champions of modeling seem to agree on, it is the need to communicate the field’s limitations, something health officials, politicians and the media sometimes ignore.

Failing to explain fully the basis for projections and the fact they could well be wrong, risks fueling conspiracy theories and COVID denialism, undermining the whole effort, said Chagla.

“We’re a society that has been so paralyzed by COVID … everybody is looking at this stuff and saying ‘Where did this come from?’ ” he noted. “You can’t go ‘Here it is, see you guys later’ … It’s really important to be transparent.”

• Email: tblackwell@postmedia.com | Twitter:

News Near Wallaceburg

This Week in Flyers