OIA and the Marsden Jacob Report

Some results of a couple of OIA requests.

On 23 September 2009, Marsden Jacob produced a short proposal for the Law Commission outlining the work it could do as part of a "Scoping Study of Optimal Taxation of Alcohol in New Zealand." The Law Commission's "tight timeframe ... precluded a competitive tendering process. ... MJA was invited by the Commission to undertake the work as it has experience in alcohol policy and other social policy issues including gambling and the role of pricing in delivering health outcomes."

Now, if you run a Google Search, you'll find Marsden Jacob Associates. Don't hit the third link on the front page: that's my prior post noting that, with respect to its review of the literature, Marsden Jacob and Associates either is incompetent at basic reading comprehension, or deliberately promulgates lies. I'd guess they're well pleased that that particular line is what's excerpted in the Google search and shows up third on the front page. But scrolling on over to their list of professed expertise on their site, you don't see alcohol anywhere listed. Hitting "alcohol" in their search box: again, nothing listed. But they have been active in the anti-alcohol campaign. Dr. Marsden presented a paper at an anti-alcohol conference also attended by Sir Geoffrey Palmer in August 2009; perhaps that's when Palmer decided they'd be best suited to produce his no-compete, non-tendered report. But, Marsden Jacob has written at least one prior alcohol report, here. Perhaps that plus a friendly chat at an anti-alcohol conference is sufficient for a no-compete contract.

The work proposed, at proposed cost of $28,800 for twelve days' work, would have been "presented in succinct, polished written drafts, but would not be intended, at this preliminary stage, for wider distribution." Dr. John Marsden's letter to the Law Commission continues:

As discussed, a study for wider distribution would require considerably more work to make it able to withstand the anticipated antagonistic and heavily resourced response.

Well, they got the "antagonistic" right.

The limited scoping study included billing for two days of literature review, to be undertaken by Gene Tunny, Senior Economist at Marsden Jacob. And so now I know who was primarily responsible for the literature review.

Sir Geoffrey Palmer's reply of 29 September notes:

I realize that a 'wider distribution' of your study may necessitate a higher degree of quality assurance, and would be comfortable considering an increase in your proposed fees to cover the additional days required for ensuring the work meets your standard for inclusion in our final published report.

Marsden Jacob received the proposed increases - for a total of $60,000 - with the final product to be delivered mid-November. But it certainly didn't take any kind of heavily resourced response to find rather serious problems in their work. Rather, a very cursory reading of their report would have been sufficient for anyone familiar with the ongoing debates here.

I subsequently corresponded with John Whitehead at Treasury, who indicated that Treasury had recommended that the report be subject to independent quality assurance.

I today received an OIA response from Brigid Corcoran at the Law Commission stating that:

Sir Geoffrey asked the Secretary of the Treasury to examine the Marsden Jacob Associates (MJA) report. The Secretary's response to this request is included in the Commission's final report along with the MJA report, as Appendix 1. The Treasury advised that 'some form of independent quality assurance would be prudent' in relation to the reliance in the MJA report on the policy-analytical framework developed at the University of Sheffield. However, the timeframe for the publication of our final report precluded the Commission taking action on this advice. Rather, we included the Secretary's letter along with the MJA report in the Appendix in order to ensure the Government was aware of Treasury's view in relation to this particular aspect of the MJA analysis. It will be up to the Government to commission a review of the MJA analysis should it choose to do so.

Recall the timeline. Law Commission gets the report mid-November and publishes its report end-April. Treasury replies 2 February. And there was no time for an independent review? It took me all of a couple of hours to note that they'd completely screwed up. The first time I saw the report was on its publication; basic professional courtesy suggests providing advance copy. Heck, basic professional prudence recommends it: who better to point out if you've completely screwed something up than the guy being attacked? The best summary of the Marsden Jacob report, with respect to its review of the literature, remains "Marsden Jacob and Associates either is incompetent at basic reading comprehension, or deliberately promulgates lies".

It's also interesting that Palmer has so frequently chosen to cite the Marsden Jacob numbers despite Treasury's noting that the report needed independent quality assurance. Perhaps Treasury should be a little less veiled in its critiques. At some point, Treasury is going to have to start being a little bolder.

Again quoting Birgid Corcoran, this time in her letter of 11 May to Roger Kerr (who sent in one of the OIA requests here noted):

The work proceeded on the basis of Sir Geoffrey's letter and the final MJA report met the Commission's requirement that it be completed to a 'publishable' standard.

An undergraduate student turning in a paper that so severely mischaracterised its sources would receive a stern lecture from me at minimum, but apparently the standards in a Canterbury undergraduate economics course are higher than the Law Commission's standard for work meant to influence policy. Interesting.

If anyone's particularly keen on seeing the whole thing, I've uploaded it to Scribd, here; I've also blockquoted above the entire response I received from Ms. Corcoran regarding independent review of the report.

The quality of policy advice in New Zealand is abysmal.