Tuesday, 10 September 2019

Rubbish statistics - Updated

UPDATE: it looks like KNZB has quietly retracted the dodgy stuff. Original post follows below, then update and comment.

I caught this rubbish floating around last week - new figures purporting to represent the amount of littering that goes on in New Zealand.

Here's the infographic that Keep New Zealand Beautiful put up to go with their report.

The numbers didn't make any damned sense on the face of it. 

Headlines talked about there being ten billion littered cigarette butts around the country. But anyone with a passing familiarity with any of the relevant numbers should have been sceptical. To a first order approximation, the government collects about $2 billion per year in cigarette excise taxes, and the tax on each cigarette is about a dollar. So that's about two billion cigarettes sold per year in total - as a rough estimate. Every one of those cigarettes would have to have been littered, for five years, for that number to make sense. Or half of them littered over a decade, and all of them persisting through to today. Does that seem plausible? I'm happy to buy that a pile of cigarette butts wind up littered, but these numbers sound implausible. 

And 395 million litres of littered disposable nappies? Suppose that each diaper is a litre. And suppose that every baby in the country goes through five per day in their first year, four per day in their second year, two per day in their third year, and none per day after that until they start voting for New Zealand First. That would be about 4000 diapers per kid over those three years. Let's round it up to 5000 to make for round numbers. Round numbers are easier. 

395 million litres of diapers, at one litre per diaper, would be the output of 79,000 children from birth through toilet training. 

There are about 60,000 children born per year. 

So every diaper ever used by every kid born over a 16 month period would have to be littered for that statistic to be true. Or half of all diapers used for every kid born over a 32 month period, or a quarter of all diapers used for every kid born over a 64 month period - and with all of the diapers persisting. 

Does that make any kind of sense? Does it seem plausible? How innumerate do you have to be to think that this is possibly true?

All results were quoted against a 1000 m2 site area and extrapolated across the area of New Zealand.

If you sample a few sites from the less than 1% of New Zealand that's urban, and then the busier parts of the rest of the country (roadsides, industrial sites, car parks, retail sites, public recreation areas and the like) and then extrapolate to the 99% of New Zealand that isn't urban (including the 78% where nobody lives), you're going to overestimate things. 

There aren't obvious problems in their site stocktakes, it's the extrapolation to the rest of the country that's a huge and obvious problem. Anybody who is half-way numerate should have been able to tell that the Really Big Numbers presented for the whole country couldn't be right. But the numbers look authoritative. And Keep New Zealand Beautiful advertises that Stats NZ helped in developing the audit (presumably the sampling, and not the aggregation - there's no way that Stats would have recommended doing this). 

And so we got reporting like this:

The project, conducted by not-for-profit organisation Keep New Zealand Beautiful, took five months to complete.

With the help of the Department of Conservation, Statistics NZ and the Ministry of Environment, two leading researchers pegged out areas across the country and calculated their litter content.

Keep New Zealand Beautiful's chief executive, Heather Saunderson, says the results are frustrating.

"The research estimates 10 billion cigarettes are littered across the country. That equates to 2142 cigarette butts per New Zealander."
Enough takeaway containers - 258,043,800 litres worth – were found to fill 25 rugby fields one metre high, while the 364,965,000 litres of disposable nappies was enough to take up 154 Olympic swimming pools.

Despite drops in smoking rates, discarded cigarette butts remained a big headache: some 10,269,090,000 were picked up, or 2,142 for every person in the country.

KNZB chief executive Heather Sanderson said, at railway sites around New Zealand, nearly 12 litres of litres of litter was being found every 1000 square metres.

"Extrapolated, that means 265,324,848 litres of illegal dumping – enough to fill 2,123 rail carriages, which if you stack them on top of each other, would be as high as 151 Sky Towers."
Bad policy comes from bad public perceptions of the true state of the world. Bad public perceptions of the true state of the world are fostered by innumerate journalists who haven't the time to think critically at all about any number presented.*

KNZB needs to withdraw its Very Big Numbers, and, if they want to have a number that might represent the country as a whole, work with Stats to find a better way of doing that. In the absence of KNZB doing that, Stats really should withdraw its imprimatur from the report.

* Unless that number comes from anyone associated with Big Industry (tobacco, sugar, alcohol - whatever). Then even accurate numbers are dismissed out of hand, because innumeracy means leaning on trusting the source because you have absolutely no clue how to judge whether a number is right or not.

Update: Thomas Lumley makes similar points.

Update 2: I was in touch with KNZB about this prior to having posted, as well as with Stats NZ. KNZB showed no interest in correcting things, so I posted on the issues.

It looks like they've now quietly retracted the bogus stats.

The National Litter Audit page no longer contains the infographic (though it's still around, unlinked, on the back-end, here). The original link to the report no longer works; this new version doesn't contain the rubbish stats.

It's good that they've fixed it, but there's been no official retraction of the dodgy parts that I can see. So it's pretty low odds that the prior news stories will put up updates correcting anything. Alas.

No comments:

Post a Comment