All of this links into broader questions about the way economic analysis is framed, assumptions made and conclusions drawn. Its the same can of worms because what ultimately matters is the quality of the argument, not the funder. In highly contentious settings like court cases, quality of argument is what counts. But a lot of policy advice does not have this wonderful attribute, and indeed there can sometimes be a strong demand from bureaucrats & politicians for the ‘wrong answer’. In such cases, supply is pretty likely and you can’t always tell whether the supplier is actively pandering to demand or just a bit hopeless.[emphasis added]Small nails this one. Thin markets mean that rubbish analysis often is let to stand: there's little market in taking sledgehammers to things like the PriceWaterhouse Coopers report on Adult and Continuing Education. Who'd fund it? And who has the time to do it for free? It's obviously completely shonky. But folks who find its numbers politically convenient keep citing it.
I’ve tried a few times, without success, to think up a workable review system that would catch such things. Ultimately, the problems are (a) a very thin market for professional economists and (b) the existence of a demand for rubbish. If you want a silly plan blessed by someone that calls themselves an economist, you’ll probably be able to achieve that aim. The way this problem is solved in other professions (medicine, law, accounting) is through professional societies that have disciplinary functions and, most importantly, barriers to entry. [emphasis added]
Maybe there is another way, but I can’t see a code of conduct having any real effect without some kind of big stick to back it up. Expulsion to the other side of the entry barrier is how other professions do it, but they have more members and more homogeneity of work, so they are more easily able to stand the fixed costs of such a system. Plus the idea of deliberately erecting a barrier to entry might not sit well with your average economist.
And pretty much anybody here can call himself an economist. Without compiling any lists, it's surprising the thin level of qualifications of all kinds of media-prominent economists. Chief Economist for X may only have a Bachelor of Commerce in Economics. But he's still cited as Chief Economist for X whenever quoted on radio. Not that having a Masters or PhD stops you from being an idiot either - it just improves the odds. We're well past the days where folks like Marshall, Keynes, Hicks, Coase and Tullock typified the non-PhD economist.
The best we can perhaps hope for is that the New Zealand Association of Economists start providing a paid refereeing service for consultancy reports.
Its worse than this. There is a lot of evidence of publication selection bias in empirical economics - results that are not consistent with priors goes unreported. Would you "disbar" someone who doesnt report what they find in the lab?
ReplyDeleteThen there are those who commit fraud - a negative sign is changed into a positive one. Some of this is published in top journals. I mean top 10 economics journals. The problem is known. No-one does anything about it.
Hey Eric, Happy New Year!
ReplyDeleteIf you think about our friends the Accountants, they're in a very similar boat with almost a certainly large demand for shoddy accounting numbers by people paying the Accountant's bills. The Accounting Society jumps on that, but the main constraint is professional ethics. And I'll suggest the main thing hampering Economics solidify into a unified profession is the inability to pronounce what "wrong" answers might look like in terms of processes followed to form policy answers or provide and economics position.
We can describe somethings as methodologically wrong therefore unethnical, like looking at costs and not benefits, or not expressing all elements of a problem on a common metric. But without looking at common economic questions in detail and proscribing a "professional" set of requirements the Econ Association will have a problem holding any shoddy to account.
Maybe that's what the Econ Association needs: a body that proscribes an "professional" approach to economic enquiry on a list of common economic issues which are at least a starter for 10. A failure to follow NZAE methodological guidelines as set by the NZAE on say, economic growth commentary, might hold the commentator up for critism in the public arena.
Just an idea ;)
Excuse me for saying this, but am I the only one having a quiet chuckle when the head of Covec speaks of quality?
ReplyDelete"disciplinary functions and, most importantly, barriers to entry"
ReplyDeleteThe "disciplinary functions" just means that the incompetent are protected by their mates. The real answer is competition. How about a requirement that policy advice be put up for publication and see what can get published and where.
"We're well past the days where folks like Marshall, Keynes, Hicks, Coase and Tullock typified the non-PhD economist."
ReplyDeleteIts not just that these guys are non-PhD economists its that most of them weren't even trained as economists. Marshall training was in maths, Keynes in stats, Tullock in law etc. But here is the point; these guys from outside the profession provided competition for those inside and thanks to low barriers to entry could enter the market and out complete those on the inside. These guys show why you don't want entry barriers.
@anon Ideally, journals would require data and code be public for replication.
ReplyDelete@James I can buy NZAE being a clearinghouse for referees. But best practice guidelines ought come from a broader field.
@Paul Agreed on potential for cliquey stuff in a thin market. I still think the refereeing could be worthwhile.
I attempted to post the following comment on John Small's post about 'Ethics for Economists':
ReplyDelete********************
“In highly contentious settings like court cases, quality of argument is what counts. But a lot of policy advice does not have this wonderful attribute, and indeed there can sometimes be a strong demand from bureaucrats & politicians for the ‘wrong answer’. In such cases, supply is pretty likely and you can’t always tell whether the supplier is actively pandering to demand or just a bit hopeless.”
You know, when I read a report from Covec called “Contribution to Government Revenue of Leaky Building Repairs”, I wondered exactly the same thing (that is, whether the supplier was actively pandering to demand or was just a bit hopeless).
********************
My comment was not published.
The New Zealand Institute of Chartered Accountants (NZICA) offers the perfect illustration of Paul Walker's comment about how “disciplinary functions just mean the incompetent are protected by their mates.” I complained to their Professional Conduct Committee (PCC) once about an accountant who attempted to steal $18,500 off us via what she called a 'capital adjustment' in a set of accounts. Her reasoning for the adjustment was contemptibly stupid – she was clearly oblivious to the absolute basics of accountancy and was essentially attempting to rewrite its fundamental principles as laid out in 1494 by Luca Pacioli. Her 'capital adjustment' was a capital adjustment in exactly the same way that a burglary represents a capital adjustment in favour of the burglar. Further, she claimed that the idea for it came to her in a dream (Peri Finnigan will confirm that, in case this seems too amazing to be true).
My complaint to the PCC about her attempted fraud and incompetence was dismissed, with the PCC refusing to tell me whether or not they considered the adjustment to be valid.
I finally (after much effort) got the police to look at it. They were nice to us (and I was grateful for that, because in all this hardly anyone was), but their opinion was that she could plead incompetence as a defence. That was fair enough, because the standard of proof required in a criminal prosecution is high.
So, if you're an economist who actively panders to demand or is just a bit hopeless, and you want protection from an aggrieved public, then membership of a self-regulating profession with a professional body and disciplinary functions is definitely the way to go.
@dragonfly: I've not read that report.
ReplyDeleteI hope your accountant wasn't Canterbury trained :>