My main reason for hating them is that they're so awfully abused by the press. It's hardly the press's fault though - if we as a profession keep producing these figures without adequately explaining (or understanding, in some cases) what they are, they'll keep abusing them.
Today's example, highlighted by Matt at TVHE: a $200m estimated "economic impact" of health disparities between Maori and Pakeha New Zealanders gets turned by the journalist into a cost to taxpayers.
The University press release:
Avoidable deaths of Māori children in New Zealand are costing the country at least 67 lives and around $200 million per year in economic impact, but greater Government spending on primary care and other key interventions could help to resolve the problem, health researchers say.The journalist version:
The public health physician said 67 Maori children died avoidable deaths every year, costing taxpayers $200 million annually.Ummm, no. Unfortunately, the Auckland University press release is about as unhelpful as possible in letting any journalist sort this out. Why? Not only do they fail to link the article, they also fail to name the journal: the study was "published in an international public health journal".
If you search through PubMed on the two authors' last names, you can find the article. It's here. But little chance a journalist will have time to do that.
What do you get if you read the article? Here's the results brief at the start:
Preliminary estimates suggest child health inequities between Māori and non-Māori in New Zealand are cost-saving to the health sector. However the societal costs are significant. A conservative “base case” scenario estimate is over $NZ62 million per year, while alternative costing methods yield larger costs of nearly $NZ200 million per annum. The total cost estimate is highly sensitive to the costing method used and Value of Statistical Life applied, as the cost of potentially avoidable deaths of Māori children is the major contributor to this estimate.So both the press release and the journalist's piece ignore that Vaithianathan and Reid produced a range of estimates with a $200m upper bound rather than a point estimate around $200m. And, the larger number relies on VSL measures. What's a VSL measure? The intangible costs of a premature death. That's only a cost to the taxpayer in the rather indirect sense that the people who are sad when someone dies prematurely may also be taxpayers, and that some of the VSL measure could be viewed (given how NZ's VSL measure is constructed) as partially being due to that premature deaths are tragic for the family. And, worse, the University press release cites the deaths as though they were something not included in the $200m.
There's other weird stuff in there. There's a measure of time out of work for grieving parents that's based around the median wage rate, but if the whole darned story is about the costs of poverty among Maori in terms of premature mortality, it's just a bit odd to use the median wage rate. But I'd hardly expect a journalist to pick that up.
Why oh why can't university press offices pumping published work done at their schools link to the darned paper and write things that won't be misinterpreted by journalists? Are they trying to confuse people? I could understand it if the article were behind a ton of subscription gates, but this one's free access.
Can we just ban economic impact studies? It's almost inevitable that the things get misinterpreted like this. And they then do far more to confuse than to illuminate.
Costs to the taxpayer of $200 million.... How long 'till somebody cites this in Parliament as a cost to the tax system? Any bets?
I like Seamus's idea - although I have my views on the answer. This is the main problem with economic impact studies, I believe - nobody actually has a reasonable idea as to the real purpose of the number at the end. To me it doesn't actually answer the all important question of whether the thing that creates the economic impact is actually a sensible idea in the first place. The common thread between all such studies is the connection to public funding - and surely the question that should be answered by any analysis worth its salt is whether the investment is justified. Economic impact figures by themselves are insufficient and say pretty much nothing helpful in the bigger picture. What is needed is a comparative economic impact of an alternative use of the government funding to give the economic impact some kind of relevance. Surely governments at all levels would be keen to see how an investment in a sporting event such as the Rugby World Cup or a V8 Supercars race, for instance, would stack up against an equivalent investment in the health or education sector? Any alternative will do! And don't get me started on whether the economic impacts actually materialise....
ReplyDeleteBoth the press release and Hill's interpretation are awful but to me the worst feature is putting lipstick on the pig. This is really a story about poverty in NZ, about solo parent families, under achievement in education and life by *parents*.. and little to do with the health authorities.. after all, if parents do not present their children for treatment, then bugger all can be done to help them.
ReplyDeleteIf you want to do this story right you have to explain that it is an *outcome* of a much larger story.. and explain the figures much better!
JC
Sounds like someone needs to do a study of the economic impact of economic impact studies?
ReplyDeleteThink of the statistical life years that have been lost writing and reading them. And the wages paid for their production...
ReplyDelete