Saturday, 31 March 2012

Stat of the week

I think I've found a winner for the next StatsChat "Stat of the Week" competition. Auckland University's Statistics Department gives a weekly prize for the best or the worst journalistic use of statistics. This is another submission on the bad side.

The Christchurch Press runs a story by Jason Krupp arguing that tobacco costs the New Zealand public health system $7 billion per year.
According to the World Health Organisation's Economics of Tobacco Toolkit, health costs attributed to smoking account for between 6 per cent and 15 per cent of national healthcare expenditure in developed countries.
In Australia, smoking costs equated to between 2.1 per cent and 3.4 per cent of gross domestic product.
New Zealand was not featured in the report but, if the results were comparable here, it would mean Kiwi taxpayers fork out about $7 billion a year to treat smoking-related diseases.
That number seems very high. Recall that New Zealand is a country of about 4.4 million people. Is it any way plausible that each and every one of us are shelling out about $1600 per year to cover smoking-related illness? That's the first thing that a numerate journalist should have thought about: a sense of scale.

Next plausibility check: how much does the government spend on the health system in total? Treasury's site is down again and so I'll have to trust in this infographic; it's from Keith Ng, so it's almost certainly correct. It cites numbers of $13.2 billion for 2011 and $13.9 billion for 2012. Does it seem plausible that smoking accounts for half of total government health expenditures? That we could double health services but for the existence of smokers?

Numbers that far out of whack demand a bit of fact checking. What would five minutes on Google tell you about prior estimates of the costs borne by the New Zealand Health System? You'd probably find the Ministry of Health's $1.9 billion estimate, but that one was predicated on the assumption that smokers would otherwise live forever with zero health costs: they didn't account for that smokers do eventually have to die, even if they don't smoke, and so most of the health costs of smoking are just bringing forward costs that would otherwise be borne by the health system even if smoking had never existed. Here's a piece from the Christchurch Press noting, and critiquing, that MoH estimate.

Next, you'd find the study by Des O'Dea, commissioned by Action on Smoking and Health, and by the SmokeFree Coalition, that concluded that smokers cost the health system some $350 million in 2005: 5% of the figure cited by Krupp. Inflation hasn't been that high over the last 7 years. O'Dea cites total figures including smokers' spending on tobacco and a bunch of other costs properly viewed as privately borne that amount to $1.7 billion: still rather smaller than Krupp's figure even though in includes a really broad assortment of costs other than those falling on the government.

And, you might find the New Zealand Cancer Society saying the total cost borne by the health system is $250 million.

And we'd also want to remember that smokers pay about a billion dollars a year in excise taxes.

I really hope that Krupp isn't based at the Press and is just part of the Fairfax stable. Because I gave a talk at The Press last year that I'd hoped would cure them of this particular kind of innumeracy.

Is it too much to ask that the Fairfax papers not invent statistics that leave their readers worse-informed for having bought their newspapers?

10 comments:

  1. "According to the World Health Organisation's Economics of Tobacco Toolkit, health costs attributed to smoking account for between 6 per cent and 15 per cent of national healthcare expenditure in developed countries.

    In Australia, smoking costs equated to between 2.1 per cent and 3.4 per cent of gross domestic product."

    He should have noticed that even these two stats are way out of kilter. 6 - 15 percent of our health bill would be about $780-1,950 million as opposed to 2.1 and 3.4 percent of our gdp which would be $3.57 - 5.78 billion.

    Where did he get $7 billion from??

    (As you note Treasury down so used figures from memory)

    ReplyDelete
    Replies
    1. With innumeracy as bad as in that Press piece, it's hard to know where to start....

      Delete
  2. very funny Eric, trying to cure Fairfax of stupidity and crass,and illiteracy, and innumeracy ; we're right behind you Eric, you go first though;
    and I think even my spelling is ok this time

    ReplyDelete
  3. My estimate would be that the cigarettes themselves cost about $1.7b (2.4b smoked a year), and health costs about the same, using the WHO report - did you check their assumptions? That's already $3.4b (although some of the sticker price pays for the health costs - don't know if they both count in GDP). I suspect they get their figure by putting a price on deaths - there are 5000 smoking deaths a year in NZ. Road deaths are priced at $4.2m. Road deaths cut an average of 45 years off your life, smoking about 15, so the deaths might add about $7b to the total cost of smoking. Halve it for caution and there's your total of $7b.
    To me it does seem plausible and not a "bad statistic of the week".

    ReplyDelete
    Replies
    1. First, there's a huge difference between all the stuff you're adding up and "taxpayers fork out about $7 billion a year to treat smoking-related diseases." That by itself is enough for a bad stat nomination.

      Second, note that the O'Dea report included all of that stuff in its final figure - the $350m I'm citing from it is just on the costs to the Ministry of Health. And it got a total figure of $1.7b. You're trying to tell me that the O'Dea report understated social costs by $5.3 billion? No way.

      Delete
    2. The big error made by the journalist is thinking that the Australian number, which covers costs to the country including costs borne by smokers themselves, is a measure of the costs to the health system. He uses that to extrapolate a cost to the NZ health system that is seriously out of line with the first WHO thing he quoted saying costs to the health system are less than 15% of health budgets; he's saying, on his derivation , that costs are 50% of the health budget.

      It's easy to conflate "social costs of X" as meaning "costs to the taxpayer". The Collins and Lapsley Report, which is almost certainly what the journalist is relying on, makes it easy to make that conflation unless you're willing to read it carefully. That's one reason that "Social costs of X" studies really annoy me.

      Delete
  4. I note that the story on Stuff is still open for comments Eric. I'm surprised you haven't seen fit to jump on there and debunk his figures :)

    ReplyDelete
    Replies
    1. Eric sent a Letter to the Editor, I think it was in Monday's edition

      Delete
    2. Ahh, good to know. Sadly I doubt his efforts, however noble, will make any real difference. People seem not to care about the accuracy of quoted figures any more, but only that they are "sexy" or capture the imagination in some way.

      Delete
    3. I put a comment there this morning, but they really really need instead to put in a footnote and correction.

      Delete