My column at Newsroom last week gave a roundup of the sessions I attended at the NZAE meetings where the results might be of interest to a broader audience.
It's ungated now, so folks can catch it there. Along with the usual band of sad old grouchy leftists in the comments section who hate economics and economists.
I think this was my favourite of all the sessions - but that'll largely be because of my own particular interests. I didn't name the presenter or the shop that did the work as they seemed to want to hold that back until the work is finally ready for public release. But credit really is due.
There was a superb presentation on problems in cost-benefit assessment, or rather in not using it, when deciding on major projects. Here we can consider ourselves lucky not at the outcome, but that someone is checking.
Economists prefer to rely on cost-benefit analysis when assessing projects – CBA. Some others like to use what’s called Multi-Criteria Analysis – MCA. On that latter kind of assessments, projects get scored across a variety of categories.
Cost-benefit assessment tries to put a monetary value on all kinds of different costs and benefits – some of which are harder than others to turn into dollars and cents. But Treasury maintains a comprehensive spreadsheet (called CBAx) listing the costs and benefits of many things, all of which then provide a standardised basis for assessment.
Multi-Criteria Analysis does not try to do that at all. Instead, a project gets a score within each category, the categories are weighted by their perceived importance, and the project gets an overall grade.
Suppose that you wanted the government to adopt your project proposal, and you knew it didn’t do well on a value-for-money basis. It would have a tough time under CBA. But under MCA, there’s a neat trick. If you add more categories for assessment, the weighting on cost declines automatically. If cost is one of two categories, each category gets 50 percent weighting. If cost is one of 10 categories, then the project’s poor ranking on cost can be outweighed by whatever other categories are added in.
Of course it is possible to require cost to have a high weighting. But it’s rarely done. And then we wind up being surprised by all of the expensive projects that get approved. Cost-benefit assessment is underrated – or, at least, MCA should require that rankings on cost carry a lot of weight. In the assessment exercise described, fewer than 5 percent of evaluated project proposals had a robust cost-benefit assessment.
I am very glad this work is being done, and I expect to provide a more detailed column when the authors are ready to release it into the wild.
In questions after the session, I noted that I've seen a few cases where boosters have tried to claim that their clearly-infrastructure proposal is really a social-type investment warranting Treasury's preferential 2% discount rate. I was annoyed that Treasury seemed utterly indifferent to that risk when they put up their proposal, and hoped that it didn't turn out as badly as I feared.
The presenter noted that there'd been a full slide on that issue that had been pulled so they wouldn't blow out the time constraint for the session. It is a real and bad issue. As expected. And something that prior better versions of Treasury would have been alert to.