Test to Exit

My column in this week's Insights newsletter notes the deficiencies in the Ministry of Health's recommendations around RAT results and ending self-isolation. 

A NUMERACY QUIZ FOR THE MINISTRY OF HEALTH

Dr Eric Crampton insights Newsletter 1 July, 2022

Suppose your middle-schooler came home with a math assignment.

She was told to measure everyone in the house and report on heights. But checking over the assignment, something seemed odd. Your 190cm partner wasn’t included.

“Oh, our teacher explained that most people are between 150 cm and 180cm. If we got a measurement outside that range, there was no guarantee that they were really that tall, so we should just ignore it.”

You’d worry about the state of numeracy among teachers, right?

Sure, the average person will be within that range. Some random person isn’t likely to be 190 cm tall. If the assignment had asked, “Is it likely that the next person you meet is 190 cm tall?”, the answer should be no – unless you happen to live with a very tall person.

But conditional on having been measured at 190cm, it’s more likely that you’re a tall person than that it’s just measurement error. If the ruler said three metres, you’d probably want to triple-check the figures. But 190cm isn’t that uncommon.

Why do I bring this up?

Back in May, I asked the Ministry of Health why they’ve been telling people to ignore positive RAT results after seven days of isolation. The Ministry replied this week. They told me that because seven days is beyond the average period of infectiousness, the current health guidelines do not require a negative test to leave isolation.

It wasn’t a cost-benefit assessment. It was a failure in basic numeracy similar to our imaginary teacher’s error – but far more consequential.

Work published in JAMA Internal Medicine in April suggested at least 60% of people with a positive RAT result six days in are still infectious.

The Ministry doesn’t seem to have checked any of that. They instead looked at the average period of infectiousness and decided positive results after that period could be ignored.

So people will be heading out, after isolation, on official advice, and sharing their Covid gift with others.

Yale Covid testing expert Dr Anne Wyllie had described the Ministry’s advice as ‘dangerous misinformation’.

I don’t know whether the Ministry’s answer reflects general incompetence, declining numeracy more broadly, or a dangerous mutant strain combining the two.

Let’s hope the revised NCEA numeracy guidelines filter their way up into the Ministry of Health before more dangerous variants arrive.

I had been in touch with the Ministry prior to this seeking, by OIA, what evidence they might have had in support of their view that a positive RAT after 7 days could be ignored if you had been asymptomatic for a day. 

Their reply came through last week:

It sounded like they'd just decided that, because the average period of infectiousness is less than 7 days, positive results after that interval could be ignored. So I followed up with a query. 

From: Eric Crampton 

Sent: Monday, 27 June 2022 4:41 pm

To: OIA Requests <oiagr@health.govt.nz>

Subject: RE: Response to your request for official information ref: H202206643

Thank you for this.

I would like to check my interpretation of one part here.

You note that “Current research indicates that it is possible for individuals to test positive from RAT tests beyond the average infectious period. Therefore, the current health guidelines do not require a negative test… to leave isolation.”

There are of course two potential reasons why someone might test positive after the average period of infectiousness has passed.

It could be that their infectiousness has passed, and the RAT is providing a positive result despite a lack of infectiousness. This seems to be the scenario your advice considers the more likely. 

But it could also be that the person with a positive RAT is one of those whose period of infectiousness is longer than average. An average will include some people whose periods of infectiousness is very short, some with middling duration, and some very long. Relatively small proportions of people would clear infection far faster than average, and relatively small proportions of people would take far longer than average. 

I do not know how you established the window for average infectiousness. Let us suppose that it reflects the period after which 95% of people are no longer infectious – for sake of illustration. If a person returns a positive result after that window has passed, the conditional probability of that person being among the 5% with a longer infectiousness period has to be relatively high unless false positive rates are very high. I had understood RATs to have relatively low sensitivity but fairly high specificity – false positives are less likely than false negatives. 

Work published recently in JAMA found, six days after illness onset, 61% of participants returned a positive antigen test while 36% produced samples that could produce viable cultures – a not unreasonable measure of continued infectiousness. If the 36% with viable cultures were all within the 61% producing a positive RAT result, then the odds of being infectious, conditional on having a positive RAT six days after illness onset, seem to be on the order of 60% - better than even. And the likelihood of not being infectious, conditional on having a positive test, would be about 40%. 

It sounds like your current advice around testing to leave isolation is based only on an assessment of the average period of infectiousness, not on the probability of being infectious conditional on a positive RAT at the end of the isolation period. 

Could you please tell me if I am wrong, and point me to the specific studies you’d rely on if so? You’ve provided a bundle of links to lengthy MoH documents spanning a wide variety of issues around the state of play in the pandemic. I cannot tell which studies you’re relying on in determining that a positive result can be ignored at the end of an isolation period, or that a negative result is not required.

I note that I am very likely to write a column on this specific issue if I do not get clearer guidance putting my mind at ease about the current guidelines by the end of the week.

I didn't hear back quickly, so I followed up:

Hi MoH,

I have drafted the following as column to be published on Friday, but I really would like confirmation that this is indeed what you have done here. If there was some more detailed risk assessment floating around in the background on the “No need for a test to leave isolation” recommendation, I would really want to know about it so I can correct or bin the column.

The draft text is appended below.

Thanks!

Eric Crampton

A numeracy quiz for the Ministry of Health

Suppose your middle-schooler came home with a math assignment. 

She was told to measure everyone in the house and report on heights. But checking over the assignment, something seemed odd. Your 190cm partner wasn’t included.

“Oh, our teacher explained that most people are between 150 cm and 180cm. So if we got a measurement outside that range, there was no guarantee that they were really that tall, so we should just ignore it.”

You’d worry about the state of numeracy among teachers, right? 

Sure, the average person will be within that range. Some random person isn’t likely to be 190 cm tall. If the assignment had asked, “Is it likely that the next person you meet is 190 cm tall?”, the answer should be no – unless you happen to live with a very tall person. 

But conditional on having been measured at 190cm, it’s more likely that you’re a tall person than that it’s just measurement error. If the ruler said three metres, you’d probably want to triple-check the figures. But 190cm isn’t that uncommon. 

Why do I bring this up?

Back in May, I asked the Ministry of Health why they’ve been telling people to ignore positive RAT results after seven days of isolation. The Ministry replied this week. They told me that because seven days is beyond the average period of infectiousness, the current health guidelines do not require a negative test to leave isolation.

It wasn’t a cost-benefit assessment. It was a failure in basic numeracy similar to the error that the imaginary teacher might have made – but far more consequential. 

Work published in JAMA Internal Medicine in April suggested at least 60% of people with a positive RAT result six days in are still infectious. 

The Ministry doesn’t seem to have checked any of that. It instead seems like they looked at the average period of infectiousness and decided positive results after that period could be ignored.

So people will be heading out, after isolation, on official advice, and sharing their Covid gift with others. 

Yale Covid testing expert Dr Anne Wyllie had described the Ministry’s advice as ‘dangerous misinformation’. 

I don’t know whether the Ministry’s answer reflects general incompetence, declining numeracy more broadly, or a dangerous mutant strain combining the two. 

Let’s hope the revised NCEA numeracy guidelines filter their way up into the Ministry of Health before more dangerous variants arrive. 

The Ministry provided a statement that could be attributed to a spokesperson - I couldn't include it in the Insights column, but I let them know I'd include it here. They write:

Since the start of the COVID-19 pandemic, New Zealand has been in the fortunate position of being able to monitor developments overseas and this has informed our pandemic response. The Ministry’s assessment is that other countries make decisions on isolation periods depending on the stage of the pandemic and the level of infection in their communities. In New Zealand, at our stage in the pandemic and to support our approach to limit the impact of the spread of COVID on the health sector, the seven days isolation for cases of COVID helps appropriately manage that risk.

We know that isolation for cases is critically important to limit spread and seven days is supported by evidence and experience of other countries. In the United States, for instance, the Centers for Disease Control and Prevention, recommended in December shortening the isolation period to five days. The change was motivated by science demonstrating that the majority of SARS-CoV-2 transmission occurs early in the course of illness, generally in the 1-2 days prior to onset of symptoms and the 2-3 days after. You can read the CDC media statement: CDC Updates and Shortens Recommended Isolation and Quarantine Period for General Population | CDC Online Newsroom | CDC

Rapid antigen tests (RATs) are useful tools by which individuals can test to see if they are COVID-19 positive. But in terms of accuracy, RATs are not without their limitations and we have been aware of this from the beginning. Our guidance to people who have had COVID-19 and who continue to have symptoms and feel unwell upon completion of their seven days of isolation is to remain at home for 24 hours after their symptoms resolve.

Our advice to the Government on the current seven day isolation period was based on evidence that there is a decline in infectiousness of the Omicron variant over time, and that in most cases, transmission occurs within seven days. The Ministry also recognises the need to achieve a balance between effectively managing the pandemic and the flow-on effect that an extended period of isolation would have on the wider economy and the health workforce.

I entirely sympathise with the notion that there has to be a balancing: you wouldn't stick someone in isolation forever. That would be crazy. And, in the absence of a full-blown CBA, a rule-of-thumb could well suffice. If someone poses risk that's comparable to the level of risk of a random-draw member of the public, continued isolation can't be warranted. 

But that requires having a gauge of what that risk is. And a person who tests positive will not have the cohort average level of risk. That person will have the average level of riskiness of people who test positive. Which will be higher. 

There are lots more studies coming out showing increased duration of shedding of culturable virus. Like this one showing that somewhere between a fifth and a quarter of people are still shedding culturable virus at 10 days. If you send someone who's infectious back to work at 7 days, rather than keeping that person home until a negative RAT, those extra couple days' work have a decent likelihood of sending someone else into isolation for 7 days. 

 

Now you're not going to find a study that perfectly matches what NZ has in place. The rules here say "Isolate for seven days (but vague about whether that starts on noticing symptoms or on a positive RAT), then, if symptom-free for at least a day, you're out". 

So what you really need is the infectiousness of people who have provided a positive RAT after having been 1-day symptom free 7 days from the start of symptoms (and 7 days from the first positive RAT). 

And it wouldn't be all that hard to find out either. We have thousands of people who turn up positive every day. Get a thousand of them to agree to take a RAT every day and to have a sample taken every day. Among the set of people providing a positive RAT at day seven, and who'd been symptom free, what proportion of samples can be used to culture the virus? It feels like the kind of thing where you should be able to have results in three weeks, and could have been started back in May.

And it really feels like the Ministry just doesn't grok conditional probabilities.