Sunday 23 May 2010

OIA and the Marsden Jacob Report

Some results of a couple of OIA requests.

On 23 September 2009, Marsden Jacob produced a short proposal for the Law Commission outlining the work it could do as part of a "Scoping Study of Optimal Taxation of Alcohol in New Zealand." The Law Commission's "tight timeframe ... precluded a competitive tendering process. ... MJA was invited by the Commission to undertake the work as it has experience in alcohol policy and other social policy issues including gambling and the role of pricing in delivering health outcomes."

Now, if you run a Google Search, you'll find Marsden Jacob Associates. Don't hit the third link on the front page: that's my prior post noting that, with respect to its review of the literature, Marsden Jacob and Associates either is incompetent at basic reading comprehension, or deliberately promulgates lies. I'd guess they're well pleased that that particular line is what's excerpted in the Google search and shows up third on the front page. But scrolling on over to their list of professed expertise on their site, you don't see alcohol anywhere listed. Hitting "alcohol" in their search box: again, nothing listed. But they have been active in the anti-alcohol campaign. Dr. Marsden presented a paper at an anti-alcohol conference also attended by Sir Geoffrey Palmer in August 2009; perhaps that's when Palmer decided they'd be best suited to produce his no-compete, non-tendered report. But, Marsden Jacob has written at least one prior alcohol report, here. Perhaps that plus a friendly chat at an anti-alcohol conference is sufficient for a no-compete contract.

The work proposed, at proposed cost of $28,800 for twelve days' work, would have been "presented in succinct, polished written drafts, but would not be intended, at this preliminary stage, for wider distribution." Dr. John Marsden's letter to the Law Commission continues:
As discussed, a study for wider distribution would require considerably more work to make it able to withstand the anticipated antagonistic and heavily resourced response.
Well, they got the "antagonistic" right.

The limited scoping study included billing for two days of literature review, to be undertaken by Gene Tunny, Senior Economist at Marsden Jacob. And so now I know who was primarily responsible for the literature review.

Sir Geoffrey Palmer's reply of 29 September notes:
I realize that a 'wider distribution' of your study may necessitate a higher degree of quality assurance, and would be comfortable considering an increase in your proposed fees to cover the additional days required for ensuring the work meets your standard for inclusion in our final published report.
Marsden Jacob received the proposed increases - for a total of $60,000 - with the final product to be delivered mid-November. But it certainly didn't take any kind of heavily resourced response to find rather serious problems in their work. Rather, a very cursory reading of their report would have been sufficient for anyone familiar with the ongoing debates here.

I subsequently corresponded with John Whitehead at Treasury, who indicated that Treasury had recommended that the report be subject to independent quality assurance.

I today received an OIA response from Brigid Corcoran at the Law Commission stating that:
Sir Geoffrey asked the Secretary of the Treasury to examine the Marsden Jacob Associates (MJA) report. The Secretary's response to this request is included in the Commission's final report along with the MJA report, as Appendix 1. The Treasury advised that 'some form of independent quality assurance would be prudent' in relation to the reliance in the MJA report on the policy-analytical framework developed at the University of Sheffield. However, the timeframe for the publication of our final report precluded the Commission taking action on this advice. Rather, we included the Secretary's letter along with the MJA report in the Appendix in order to ensure the Government was aware of Treasury's view in relation to this particular aspect of the MJA analysis. It will be up to the Government to commission a review of the MJA analysis should it choose to do so.
Recall the timeline. Law Commission gets the report mid-November and publishes its report end-April. Treasury replies 2 February. And there was no time for an independent review? It took me all of a couple of hours to note that they'd completely screwed up. The first time I saw the report was on its publication; basic professional courtesy suggests providing advance copy. Heck, basic professional prudence recommends it: who better to point out if you've completely screwed something up than the guy being attacked? The best summary of the Marsden Jacob report, with respect to its review of the literature, remains "Marsden Jacob and Associates either is incompetent at basic reading comprehension, or deliberately promulgates lies".

It's also interesting that Palmer has so frequently chosen to cite the Marsden Jacob numbers despite Treasury's noting that the report needed independent quality assurance. Perhaps Treasury should be a little less veiled in its critiques. At some point, Treasury is going to have to start being a little bolder.

Again quoting Birgid Corcoran, this time in her letter of 11 May to Roger Kerr (who sent in one of the OIA requests here noted):
The work proceeded on the basis of Sir Geoffrey's letter and the final MJA report met the Commission's requirement that it be completed to a 'publishable' standard.
An undergraduate student turning in a paper that so severely mischaracterised its sources would receive a stern lecture from me at minimum, but apparently the standards in a Canterbury undergraduate economics course are higher than the Law Commission's standard for work meant to influence policy. Interesting.

If anyone's particularly keen on seeing the whole thing, I've uploaded it to Scribd, here; I've also blockquoted above the entire response I received from Ms. Corcoran regarding independent review of the report.

The quality of policy advice in New Zealand is abysmal.

7 comments:

  1. Hi Dr Crampton,

    Interesting stuff. At $2800 per day (plus GST), these guys are approximately double the most expensive professional per diem I've seen in my travels. Also, I'm pretty sure there's something about govt contracts for services tenders greater than $50k having to go to GETS.

    When I was contracting, it was extremely hard to get a nose in for contracts for this type of work. Its even worse to hear they did a crap job.

    *sigh*

    ReplyDelete
  2. That fee covered three of them, though two of them were listed as providing oversight and support.

    I know nothing of this GETS of which you speak. Would it cover cases though where the initial proposal was for well under that amount, then expanded?

    ReplyDelete
  3. GETS is the Government Electronic Tendering System - http://www.gets.govt.nz/default.aspx?show=HomePage

    There are similar sites in Australia: https://tenders.nsw.gov.au/nsw/?event=public.home

    GETS was set up by the Ministry of Economic Development back when Jim Anderton was running its show. It was fantastic for small businesses because it gave us an "in" into contracts which we wouldn't otherwise have had a chance to pitch for. I used to contract with another consultant (www.wiltshirehogan.co.nz) and we'd gotten one contract from there. But its real value was in sharpening up our Expression of Interest process and our marketing overall.

    The nature of the contracts on there has dwindled over time. There used to be some real good economic projects listed there. Not much now-a-days.

    ReplyDelete
  4. Hmm, BabelFish translated the above as:

    "The world word of praise, wants like the sponge to meet the water to hold firmly"

    Deep

    ReplyDelete
  5. Looks like an easy way to negotiate an extra $30k.

    It certainly looks to me like Palmer was 'getting what he wanted'. The extra $30k was due to the expected antagonistic response (expected before the work had even been done). Why would you expect this if you also expected your work to be independent, well balanced, well researched etc?

    If this isn't a 'reports to order' smoking gun, I don't know what is.

    ReplyDelete
  6. @Matthew: Someone from the anti-alcohol side would say that a fair and balanced report would attract well-funded antagonistic responses from the alcohol lobby. It's not implausible on the face of it.

    ReplyDelete