This post is largely directed to any of this blog’s readers who perform economic impact analysis. Too many years ago, my Honours research paper was a study of the economic impact that the Christchurch Arts Festival had on Christchurch. The festival had asked for someone in the Department to undertake the analysis, but they weren’t prepared to pay anything, and so my supervisor Ken Henry (now Australian Secretary of the Treasury) agree to take it on as an Honours research topic.
I followed the usual methods employed for these kinds of studies: I circulated questionnaires at various concert venues asking patrons to answer some questions designed to find out if they were locals or visitors, if the latter, whether they came to Christchurch because of the festival, what their festival-caused expenditures were, etc. and then threw the results into input-output tables to calculate the usual multipliers.
While I did present the results following the standard Keynesian template, my base numbers were more reasonable. I assumed that additional spending by local residents would mostly have crowded out other spending rather than savings (in effect, being true to the fixed coefficients assumption of input-output analysis), and further presented results in which it was assumed that there was no Keynesian deficiency of demand that wasn’t being addressed by monetary policy. Needless to say, the festival chose never to release the report that Ken and I wrote for them!
But even fully stripped of the Keynesian assumptions, there was still the results arising from what was called the export-base model. This was the impact on Christchurch that arose from the “export” money being brought into the city by visitors from outside whose presence here was caused by the existence of the festival. This made me uncomfortable at the time, but I never was able to fully articulate in my mind whether export-base modelling made sense or not.
I have recently had cause to think about this kind of analysis again, and I don’t feel any the wiser. Let’s say someone does an economic impact report and concludes that a particular activity generates $10m of economic activity in a region, not due to any Keynesian effect in which idle resources are brought into productive use through stimulating aggregate demand, but due to bringing expenditure into a region. What is the question to which $10m is the answer? I can see that it is probably an ordinal measure of something, so that if $10m of economic activity is a good, $20m is probably better, and if $10m is a bad, $20m probably worse. But what does $10m mean as a cardinal number? Is it a benefit that can be compared to a cost in a Kaldor-Hicks cost-benefit framework, so that, say, one could conclude that public expenditure funded of up to $10m would be justified in order to secure the activity? This can’t be right, as it ignores the opportunity cost of the resources that produced the goods and services sold for $10m.
For there to be any case for public expenditure to secure the $10m of economic impact, it seems that economic impact studies of this kind must be implicitly appealing to either some notion of a public intermediate good or second-best analysis, but it is never clear to me just what the assumed failure is in these analyses. And in any event, even with such market failures, the net benefit of the public expenditure would not be $10m.
So here are two questions for those of you who do economic impact studies. First, when you calculate a number for an economic impact, do you assume that there is any underlying market failures such that the “impact” is, in fact, a benefit. And second, what exactly is the question to which the reported dollar number of impact is the answer?
I think Canterbury should run an honours course on conducting impact studies, given a lot of places grads end up working have them run these studies.
ReplyDeleteI don't do studies of this nature (or any economic studies at all). As is usual, I won't let that stop me having an opinion.
ReplyDeleteI agree that the $10 million of economic impact isn't a number comparable with the expenditure. But it is perhaps an indicator, as the $10 million should be able to be massaged so as to generate a benefits number. For example (and here I use a national example, as that's easier to point to imported benefits for):
- $10 million of economic impact, if that impact is largely spending, means $1 million of GSt
- $10 million of economic impact must logically mean some number of jobs if we assume no crowding out
I think the problem here is that these studies often quote the economic impact number without finishing the analysis. The next step is to turn that number into some measure of the benefits of that economic impact.
@Anon: Given resource constraints, we'd need to can a course to put it on. Suggestions?
ReplyDelete@Paul: GST only comes in if we're drawing in foreign visitors; otherwise, the counterfactual has folks spending money on other things. Your jobs assumption requires that there's employment slack that isn't optimally being addressed elsewhere. But what numbers I've seen suggest that folks hired tend to be drawn from other employment rather than from the unemployed. And for big spending programs like stadiums, it comes out a net negative.
@ Anon: I do cover cost-benefit analysis in my Honours welfare course, but the course is about the abstract ethical underpinnings of CBA, and not the nitty-gritty of how to abuse it to produce a large number for a client.
ReplyDelete@ Paul and Eric: Even if the economic impact came from oversease visitors (which I think was Paul's example), there would still only be a GST benefit to the extent that the resources used to produce the goods and services sold to foreigners would not have produced as much taxable output in the absence of the expenditure that lured those visitors here.
This is where you need some notion of a public intermediate good.
I also do a week on Cost-Benefit in my Econ 224 course, but it focuses on "here's what it can do, and here are really dodgy things to watch out for that folks will use to skew the numbers".
ReplyDelete@Seamus: Granted.
"@Anon: Given resource constraints, we'd need to can a course to put it on. Suggestions?"
ReplyDeleteI'm not sure that macroeconomics is all that useful anymore...
Good questions Seamus.
ReplyDeleteI think of economic impact analyses (EIA) as the answer to the question: by how much will/did local GDP increase? EIAs should be a measure/estimate of the additional value added in the local economy contingent on some event/stimulus, so (done properly) the economic impact is net of the costs of production.
The existence or otherwise of market failure is a separate issue, because such impacts can occur without government intervention. However there are often cases where local governments feel themselves to be competing with other jurisdictions for events (eg the Ellerslie Flower Show in ChCh). In such cases, the EI is an *upper bound* on what could reasonably be paid to event organisers in return for delivering the event.
As usual (when translating from concepts to the real world) there are important caveats such as:
1. It is often difficult to know whether there is in fact a market failure. For example a promoter could be willing to run an event anyway, even without a payment. So caution is required.
2. Obviously, *any* such payment redistributes wealth from ratepayers in aggregate to the much smaller group of local residents that has claims on the additional value added (note the claimants will usually include some non-residents that own local productive capacity). So, just like all other council spending, payments of this nature should be thoroughly tested against other potential uses for the funds.
3. Mobile capital tend to move. So (unless there is deal-specific investment) don't think that by attracting an event this year, you will also get it next year (eg flower shows, where I wouldn't be surprised to see "the Ellerslie Flower Show, formerly at Christchurch, now at Nelson").
I agree Eric. My point is that economic activity is an interim measure, it is a pointer to something else, and getting to that something else needs more massaging. That something else may be negative or zero.
ReplyDeleteIt doesn't invalidate the economic activity number - it just means that we need more data or analysis to interpret the number.
I fully agree with you that most of this stuff is just pure boondoggle. But, if we're going to start getting serious about the question of why governments do it, we need to move out of the realm of economic theory and into political theory. The short answer is that it wins them votes - clearly taxpayers like their money being wasted in this way.
If you then put that context around the economic analysis, you realise that the job of the analyst isn't to shed light on whether or not it is a good thing. The job of the analyst is to give a large sounding number framed by some jargon, allowing politicians to tell taxpayers that it's all right without actually having any evidence of that at all. Now a politician can say "xyz research inc have done a study, and found that our expenditure on this event will create umpty zillion dollars of economic activity." It's a meaningless statement (as you've shown above), but it makes the taxpayers happy.
So if your role is to make your students employable, then you should teach them how to write reports that are easily able to be misinterpreted, through using jargon or obfuscation such that the procurer of the report can misuse it to their heart's content.
If your role is to seek truth, then you should teach them that allowing people to use their work in that way is immoral. Lucrative, but immoral. I guess a truly inspired lecturer would teach them both, and let them make their own choice - they can know what's right and still choose not to do it.
:-)
Just to clarify a bit, if EIA is done properly, it has a meaningful interpretation. It is an estimate of something interesting.
ReplyDeleteBut to use it sensibly, you need to know what it is and isn't.
EIA is not a social cost-benefit analysis, a market failure analysis or (to paraphrase Baumol) a cure for baldness.
Like soft drink, EIA is popular and dangerous if used inappropriately. Maybe we should demonise it and/or regulate its supply?
@John: I've been doing my best on the demonisation front, at least for the dodgy ones.
ReplyDeleteMy biggest worry is that most folks wanting these don't want a sound analysis: they want a big number that can be used for political purposes in justifying a predetermined policy. And it's bloody hard for the external observer to tell the difference between that kind of number and one produced using sounder method.
And, even one done soundly does get put to inappropriate purpose. It's awfully easy for the press or others to conflate "increase in GDP" with "net benefit".
Good problem definition Eric.
ReplyDeleteBut the solution surely has to focus on educating the users of economic analysis, which includes the general/voting public.
Exposing shonky supply is a great tactic in this war. All I'm saying is lets make sure that in doing so we distinguish the enemy (crap analysis & its purveyors) from the analytical tools/concepts they might deploy (which need to be understood, not banned or demonised).
I laughed out loud at some of the comments here, especially those of Eric and PaulL. I work for a government in Canada and these sorts of "economic impact studies" cross my desk all the time.
ReplyDelete"My biggest worry is that most folks wanting these don't want a sound analysis: they want a big number that can be used for political purposes in justifying a predetermined policy."
Oh God yes, this exactly :)