Friday, 14 September 2012

Critique and the customer

If the point of an economic consultancy report is to convince serious officials at Treasury, the Reserve Bank, or the relevant Ministries of the economic case for something, and if the consultancy report has serious errors, you can do good by loudly critiquing it if it is wrong. Those who need to know whether the report is right will check the critiques and, if truth-seekers, will update. I understand that some of the serious folks at the Ministry of Health have strongly updated their views as to the appropriateness of Cost of Illness studies around alcohol consequent to critiques, very correctly noting that cost-effectiveness is the better measure.

But if the point of an economic consultancy report is to excite the hooples and to give a sciency justification for whatever some bunch of rent-seekers is trying to push, then critiquing it loudly only serves to provide advertising: this firm will produce the big shonky but sciency-looking number that your industry needs to get public support for whatever you're trying to push.
"Noise made, overtures to outside interests and enlistment of the hooples’ participation is what this situation demands." Al Swearengen, Episode 20, "Childish Things", Deadwood.
If that's what the reports provide, critiques might make things worse by letting the interests know who's best at exciting the hooples.

Want to show that your industry is the key linchpin for the regional economy? Want to show that your party's policy is the one for advancing economic prosperity? Yelling about how a particular consultancy produces numbers that are great for propaganda but bad for policy only serves to fuel demand where the ultimate audience are the hoopleheads.

What else can you say about a consultancy report that includes this in the concluding remarks?

The biggest cost will be loss of face for the “Mainstream Economists” especially the Bank economists, who have continually told us that this is the panacea: There Is No Alternative.  Their experiment will be at an end.
The audience isn't Treasury, the Reserve Bank, the boffins in the Ministry of Finance, academic economists, or anybody serious. It isn't you, Matt, as correct as your critique may be. Neither is it you, Paul: I watched you writing your critique as I went meta.  It's the hooples. And it's the reason that the New Zealand Economic Association really really needs an annual awards ceremony for the worst piece of economic consultancy work produced in the country every year.

So that the hooples can better be advised of the reputation of certain outfits.

6 comments:

  1. In your second, fourth, fifth, and sixth paragraphs I believe you've misspelled "conslutancy".

    ReplyDelete
  2. "The audience isn't Treasury, the Reserve Bank, the boffins in the Ministry of Finance, academic economists, or anybody serious. It isn't you, Matt, as correct as your critique may be. Neither is it you, Paul: I watched you writing your critique as I went meta. It's the hooples."


    I agree. But surely that makes it all the more important that the shortcomings of the report be made public so that even the hooples get to know about them.

    ReplyDelete
  3. You're probably right at current margins, but probably only because anybody who needs to know from whom to source this kind of work already now knows it; for that, I mostly blame me.

    ReplyDelete
  4. The hoopleheads DO need to know that certain outfits have a bad reputation, but its a vain hope that some sort of industry-insiders anti-award would make them aware of the fact.


    There is a difference between insiders having a bad opinion of a firm and TELLING outsiders about it, and the outsiders forming their own opinion based on critique after critique, and fisking after fisking.


    They may not understand the details, but they do appreciate the tone.

    ReplyDelete
  5. Critiques are good but an award would highlight bad firms and draw some (unwanted) attention towards them.

    ReplyDelete
  6. I have never heard a media outlet say, ever, "But X's work has often been subject to strong methodological critique from within academia, from other consultants, and from Treasury officials." An award MIGHT help on that side.

    ReplyDelete