And Treasury weighs in

And so it seems that BERL also has a different "world view" from Treasury. Update: the original article had a minor error, now fixed and noted at NBR; corrected version now quoted below.

I'm very surprised that a very senior Treasury official has come out with a statement that says, in essence, that the Law Commission's reputation is at stake if it chooses to base policy recommendations on the BERL report.

The Berl report into the social costs of alcohol for the Law Commission is work that doesn’t look like it meets the “normal standards you would expect”, according to Deputy Secretary of the Treasury Peter Bushnell.

We of course agree.

“I think the points they’re making are sound about adding the costs of production into the cost of it, and not counting any benefits. In a market if you’re selling something that people are prepared to pay for, then they’ve at least got that much benefit, otherwise they wouldn’t have bought the stuff. So if you exclude the benefits then you’re clearly only looking at one side of the story.”

There is no defensible argument for setting benefits equal to zero.

“I can see the point being made in the article -- it looks pretty shonky”, said Dr Bushnell, “I think the fact that some work’s done that academic review says is pretty shonky is a problem by itself. But the question is what’s the best way to solve that?”

He agrees that an alcohol cost analysis Treasury did in 2002 is more consistent with Dr Crampton and Mr Burgess’s findings than the Berl report, which was jointly commissioned by the Ministry of Health and ACC.

However, the mere fact Law Commission president Sir Geoffrey Palmer is seeking out economic advice is positive, “because in the past lawyers often assumed that economics had nothing to do with it.”

That said, the onus should be on the Law Commission to be more rigorous Dr Bushnell said.

“Geoffrey’s reputation is reduced [if] he’s putting weight on something that actually doesn’t stack up. So the Law Commission ought to ... build in processes that give adequate QA and so on.

“What we’re saying is it’s your reputation that’s at risk here. It doesn’t reflect well on the Law Commission if it... backs [work], that doesn’t have a sound basis.”

I'm rather sure that Geoffrey Palmer would have seen, and utterly ignored, my early critique of the BERL report.

Dr Crampton goes further. He believes the Berl report belies a more general problem with economic consultancy reports, in that there needs to be somebody looking at the Requests For Proposals (RFPs) that a ministry sends out, and checking the results when they come in.

“We really need to have Treasury vetting RFPs going out from the Ministries checking that they’re spec’d up to actually measure what they need to measure, and Treasury vetting the reports coming back. Absent that, nobody does it, folks take reports at face value and policy is not formed on sound bases”, Dr Crampton said.

But there are two main reasons nobody does this: the primary one is a lack of resources.

“Treasury just doesn’t have the resources to look at every RFP that goes out and then check the results when they come back”, Mr Bushnell said.

I agree completely. Treasury should be better resourced for this kind of work. The costs of bad policy seem highly likely to outweigh the costs of improving policy.

The secondary one is the need to maintain the independence of bodies such as the Law Commission or Commerce Commission.

“There’s sort of a question as to what are the processes that respect that independence, which is what you set them up for, but also ensures that you’ve got enough rigor and quality in the advice”, Mr Bushnell said.

“But I don’t think we can expect to have a body set up that’s going to examine every economic claim made by any organisation in the public sector.”

Sir Geoffrey was overseas when contacted by NBR, and has declined to comment on the matter thus far.

Again, it was MoH and ACC that put out the report, not the Law Commission. I'm not sure that MoH needs to be independent of Treasury oversight. I'm also not sure what the best institutional solution is to prevent messes like this in future. It's a tough problem.