Sir Owen Glenn's family violence inquiry has stumbled again, producing a $7 billion estimated cost of family violence based on the mis-reading of a key research paper.
A report by economist Suzanne Snively and Wellington theatre student Sherilee Kahui, published by the inquiry yesterday, said family violence cost New Zealand between $4.1 billion and $7 billion a year - up from Ms Snively's last estimate in 1994 of just $1 billion.It looks like they attributed the full high-end costs to the greater number of people experiencing more moderate forms of abuse. This isn't to diminish that experience, but there are gradations of experience and gradations of cost.
Inquiry spokeswoman Marie McNicholas declined to comment on the latest mistake and referred questions to Ms Snively. Ms Snively said the data was prepared by Ms Kahui.
Ms Kahui said the $7 billion "high-end" figure was not in an early version of the study, which initially included only the "low-end" estimate of $4.1 billion and what is now described as a "moderate scenario" of $4.5 billion.
The high-end estimate was added after experts in Auckland and Wellington said they believed the true domestic violence victimisation rates were higher than the "moderate scenario" rates of 18.2 per cent for women and 1.9 per cent for men.
"We were struggling to find empirical evidence of an estimate that would be higher than 18.2 per cent," Ms Kahui said. "So it was about finding something higher."Jeepers. Kudos to Ms. Kahui for pointing out that there was a quest to find a big number.
I note that Suzanne Snively also authored the PWC report on Adult and Continuing Education that estimated very large benefits from Adult and Continuing Education by assuming that anybody taking a night cooking course would be 50% less likely to commit any crime.
Congrats too to Simon Collins for catching this. I didn't have time to look through the report yesterday when I saw the big number floating around; nice to see that others have started sniffing out this stuff.
A very brief look at the link to the inquiry you provided didn't confirm, but I'm just wondering if a more generous interpretation of the quotation might be:
ReplyDelete"We have good reason (some data/estimates/some experts' comments?) to believe that true victimisation rates are significantly higher than 18.2% and 1.9%. Therefore in order to most accurately assess the cost of this level of domestic violence we were after empirical evidence that had attempted to attach a cost to the true rate (i.e. a rate higher than 18.2%). We couldn't find one so we thought a decent estimate would be to take the 'high end' cost attached to 18.2%, a victimisation rate much lower than the true rate."
There may be good reason to place more trust in other forms of estimation, but it doesn't strike me as wholly unreasonable or wilfully misleading/over-estimating (if there is good reason to believe true rates are higher).
It's possible, but isn't it then better to mung in the best guesstimate on true victimisation rates instead?
ReplyDeleteMy previous reply got lost in iPhone world. But it was the same point as Eric. They are estimating AxB. There is reason to think that the estimate of A might be too low, but they don't know by how much. So they go with an estimate of B that is known to be too high to compensate. (Cue Hayek's Nobel address on the scientistic fallacy in Economics.) Far better to present sensitivity on A. Based on the numbers in Eric's post, it seems that their high-end estimate is equivalent to guessing that the true rate of domestic violence to women is 31.1% rather than 18.2%. It would be more honest to report that as the assumption underlying the high-end estimate, and let people decide whether that is a reasonable guess at the true value.
ReplyDeleteLindsay Mitchell also points out that the previous 1995 estimate had a range of between 1.2 and 5.8 billion. "Big number bias" aside, the costs don't seem to have risen in real terms since 1995.
ReplyDeletehttp://www.lindsaymitchell.blogspot.co.nz/2014/11/glenn-report-says-family-violence-costs.html