For the second time this year, Jeff Vogt will have to return to an all-takeout menu at JV’s Downtown Bar and Grill in Waterloo, a southern Illinois bedroom community popular with people who commute to downtown St. Louis.
That’s because the percentage of COVID-19 tests coming back positive in the metro-east surpassed 8% for three consecutive days on Aug. 18 — the official trigger for tougher restrictions.
It means layoffs and lost revenue at the restaurant known for its brisket and homemade sides. Yet Vogt admits he doesn’t really understand why positivity rates seem to hold such power.
“I try to read and educate myself and listen to the people who I think understand it,” Vogt said. “I don’t get it. I wish I understood it better.”
Despite its pivotal role in tracking the virus and informing decisions about community response, scientists don’t agree on how well positivity works as an indicator of where the virus is headed in a given region.
No one in the scientific community or state and local government says COVID-19 is anything less than a deadly problem. In the metro-east alone, 52 people succumbed in August. Hundreds more were infected and dozens hospitalized, with the long-term effects of the disease still unknown.
But the positivity rate is not a useful number without adequate testing, said Sarah Cobey, an associate professor at the University of Chicago who specializes in ecology and the evolution of pathogens. That is the case in some metro-east counties.
“I have been very critical of their use of this metric and basically all of the metrics they’ve put forth so far because they are not scientifically founded,” said Cobey, who has advised the Illinois Department of Public Health on tracking. “They’re roughly right, but they’re not metrics you want to hang your hat on.”
Cobey said the state wanted to use a metric the public could understand and calculate on its own.
“We said, ‘You’re going to be losing scientific accuracy and probably credibility in the long run if you start using these other things,’” Cobey said. “They’re pretty adamant that actual science is too much.”
There’s no denying its simplicity. Divide the total number of tests by the number of positive tests and multiply by 100. Epidemiologists consider 5% an indicator that the virus is mostly controlled in a region.
Other scientists say the positivity rate is reliable and one of the best and only real time indicators available.
“You have to rely on things that are actually happening in real time and that computers can grab,” said David Dowdy, associate professor of epidemiology at Johns Hopkins University. “The number of tests that are performed and the number of positive tests that are detected are two pretty easy indicators to find.”
The perfect metric to capture the spread of COVID-19 doesn’t exist, the scientists agree. But political pressure to find one mounted last week after Democratic Gov. J.B. Pritzker confused the public and regional leaders by treating two Illinois regions differently in implementing new restrictions.
It’s time to revisit the positivity rate as the gold standard for restrictions, said state Sen. Paul Schimpf, R-Waterloo.
“We need to recognize that there is some concern about its accuracy.”
WHAT IS THE POSITIVITY RATE?
Vogt, the Waterloo restaurant owner, isn’t alone in feeling perplexed by the overall concept. Even some of the state’s top epidemiologists are stumped as to why the state chose 8% positivity as the threshold for taking action to slow a resurgence of the virus.
“To be honest, there is no scientific explanation for those numbers,” said Jaline Gerardin, assistant professor of preventative medicine at Northwestern University who has worked with the state on tracking the virus.
IDPH chose 8% as the threshold to help avoid potential confusion caused by rapidly changing restrictions, IDPH spokesman Cris Martinez said.
Once restrictions are activated by the 8% rate, a region has to show 6.5% test positivity or below for two weeks before the rules are relaxed. Setting the threshold for restrictions a full percentage point and a half higher allows for some wiggle room so rules aren’t changing every day.
“It is not reasonable to expect people and business to follow control measures that change every few days,” Martinez said.
Positivity can be useful in showing two things: whether enough testing is being done and how widespread the virus is. The two go hand-in-hand. When large portions of a population are tested, the positivity rate can be a reliable indicator that all, or close to all, infections have been identified, public health professionals say.
But positivity’s utility only goes so far, Gerardin said. Because the number of tests performed changes in any given day, week or month, it’s difficult to measure across time.
A lot of testing can drive positivity down even as more cases are discovered, but “it’s not intended to be a true reflection of how many people in the general population have COVID-19,” said University of Illinois physics professor Nigel Goldenfeld, a key figure in Illinois’ pandemic response.
The problem in the metro-east is insufficient testing, said St. Clair County Chairman Mark Kern, and that those who are tested are likely symptomatic. The result is enormously high percentages that don’t accurately reflect how the virus is spreading.
For instance, Bond County in the metro-east on Tuesday reported a daily positivity rate of 55.6%. That’s because only 18 tests were conducted and 10 came back positive, Kern said.
High positivity drags down the cumulative rate for the metro-east region, which IDPH calls Region 4 and defines as Bond, St. Clair, Madison, Monroe, Washington and Randolph counties. When the overall rate surpasses the 8% threshold, the entire region sees restrictions even if just a few counties are the cause.
Lawmakers such as Schimpf and state Rep. Charlie Meier, R-Okawville, have urged the governor to use hospitalizations instead. Hospitalizations can be a good indicator of COVID-19’s presence, Gerardin said, but the data often lags behind by 10 to 17 days. Ideally, epidemiologists would have more up-to-date estimates of how transmission rates are changing.
Regardless, relying on the positivity rate that doesn’t reflect the real picture is unfair to businesses such as JV’s in Waterloo, Schimpf said.
“(Pritzker’s) plan relies on the positivity rate as a trigger for actions, even though we’ve clearly seen that in Region 4, the positivity rate does not contain sufficient accuracy or insight to meaningfully assess the spread of COVID-19,” Schimpf said. “ … It’s kind of unfair that a businesses like (JV’s) is being evaluated on a metric that includes stuff that they have no control over whatsoever.”
OTHER METRICS FOR TRACKING COVID
Economics aside, incomplete information about the virus’ spread makes it difficult for public officials to respond effectively, Cobey said. There are other metrics for tracking how widespread the virus is in a community and for making decisions.
One of them is a sentinel surveillance program that would track a group of people with similar characteristics, such as showing the same severity of symptoms for around the same period of time, and who are easy for health care professionals to reach for testing, Gerardin says. Scientists could glean information about the virus’ prevalence from the sample population.
“The advantage of that is that it is consistent,” Gerardin said.
Bringing the data together for a sentinel population would be “really challenging,” she added, because it means compiling test results, surveys and other details. Additionally, collecting a random sample that represents the population takes time, said Dowdy, the Johns Hopkins professor.
“You can’t do that on a daily basis,” Dowdy said. “It’s hard enough to figure out how to get a random sample of people. It’s really hard to do a good study that’s representative of the population, and by time you’ve done that, it’s already a month or so gone.”
Positivity can expose a lack of testing, but there’s another metric for that too, according to researchers at the volunteer-operated COVID Tracking Project. Flipping the positivity rate fraction can show how many tests it took to reveal a positive case. This provides a better indication of whether testing is sufficient, the researchers say.
If it took 20 tests to find one case, it could indicate that more than just symptomatic people are getting tested. If the number of tests it took to find a positive is low, it could mean there are more undetected infections and reveal the need for more testing.
In a place like Bond County where few tests are performed at all, neither ratio works. There’s no way to determine how widespread the virus is in a community when there’s not enough testing.
Even an area with plenty of testing could have a misleading positivity rate, Dowdy said.
“If you have an area that does a lot of testing on a large part of their population, that’s going to drive down their percent positive even if they have a reasonable amount of transmission in the community,” Dowdy said. “There’s not a single number that you can say, ‘This indicates uncontrolled transmission.’”
The governor’s office and IDPH have shown a willingness to work on the data scientists thing is more accurate, Cobey said. Meantime, she hopes the public doesn’t take criticism of the state as an effort to sabotage genuine public health efforts.
“Some people will take any criticism of the response so far as an excuse to behave totally recklessly,” Cobey said, “and I don’t want that.”