Thursday 13 March 2014

Discounting Cycling

The Greens recommend shifting some transport funding from lower value roading projects towards bike lanes near schools. It doesn't seem like a bad idea, so long as NZTA does proper modelling of effects on traffic flows and doesn't wind up putting too great of constraints on trunk routes that happen to go by schools.

But I wish they wouldn't have put up their indicative cost-benefit figures. Here's the Greens:
New research from the University of Auckland shows the kind of infrastructure the Greens will build for walking and cycling will produce up to $4 billion of benefits over 40 years for a $200 million investment - a 20 to 1 return.19
Ok. First off, 20:1 benefit-cost ratios are immediately eyebrow-raising. If something's that great, why hasn't it already been done? My first thought was that somebody's gone and rolled up forty years of estimated benefits without any time discounting. So, let's check Footnote 19. It's this.* And what do we find at page 10?
To make the findings policy relevant, we used the national transport agency’s methods (New Zealand Transport Agency 2010) to calculate indicative benefit-cost ratios in New Zealand dollars (NZD) for each scenario compared with baseline (summed net benefits divided by infrastructure costs). Infrastructure costs were not inflated and, in contrast to the transport agency’s methods, neither did we discount future benefits and harms. [emphasis added]
Where a bunch of the benefits come from estimated future health improvements and mortality reductions, choosing not to discount future benefits at standard discount rates kinda determines their results for them. How can they say that this project is higher value than other NZTA projects when standard NZTA projects discount future benefits at standard discount rates? Again from Macmillan et al paper:
Table 3 illustrates the estimated cumulative costs and benefits to 2051 of the four policy interventions compared with the business-as-usual simulation. All interventions exhibit positive benefit-cost ratios, ranging from 6 to over 20 dollars saved for every dollar spent on bicycling infrastructure. The largest savings come from reductions in all-cause mortality due to physical inactivity.
If the Greens are happy with this kind of no-discounting cost-benefit analysis, I have a proposition for them. For the low low cost of $1,000,000, I will pay them $25,000 per year for the next eighty years. If we roll up the 80-years' benefits, that's $2,000,000: a 2:1 benefit-to-cost ratio! Sure, it's not 20:1, but it's still pretty good. Two is bigger than one. Right?

It wouldn't be hard to convince me that this could be a good policy. But hanging it on this kind of analysis sure isn't convincing.

* Alexandra Macmillan, Jennie Connor, Karen Witten, Robin Kearns, David Rees, and Alistair Woodard. 2014. "The Societal Costs and Benefits of Commuter Bicycling: Simulating the Effects of Specific Policies Using System Dynamics Modeling." Forthcoming in Environmental Health Perspectives.

20 comments:

  1. Looking beyond the discounting, which seems somewhat suspicious, I'd also be concerned about the bulk of the benefits being attributed to a diffuse public good, such as reductions in all cause mortality.


    My concerns are:
    - is the calculation optimistic, or using realistic values?
    - what are the alternative policies that could give similar health benefits using similar assumptions?


    Looking at the benefits to the community, I see:
    1. Reduction in fuel usage ($1.2B). Is this a legitimate public benefit, or is this a private expenditure? What about the costs of the bicycles and helmets that need to be purchased? The bike maintenance? Is there a time cost? This just sounds a bit one sided


    2. 1325 fewer deaths, a combination of avoided road deaths and better health leading to fewer deaths.


    3. 3600 fewer days lost of work due to ill health.


    The footnote points to this study: http://ehp.niehs.nih.gov/wp-content/uploads/122/2/ehp.1307250.pdf (as an aside, it'd be nice if the Green's footnotes were hyperlinks, or even copy and pasteable. Retyping them is annoying).


    On page 19 of that study they briefly discuss the calculations relating to all-cause mortality reductions through increased physical activity. The description is not overly detailed, particularly for something that appears the largest driver of the positive benefits. I certainly don't see any calculations that suggest they've considered replacement of one type of exercise with another (e.g. I currently run in the mornings, I'd probably bike to work if there was a cycle lane, but I'd probably not run on the days I biked). I think this portion of the study is perhaps suspect, and it would benefit from some consideration of substitution impacts, and also what other policy initiatives might generate physical activity benefits at lower cost/impact.


    Like you, I'd like to live in a city more like some European cities with more accessible cycling. I'd be open to that consideration. But I'm very sceptical that it can be achieved without substantially impacting those who (for whatever reason) need to drive, nor without substantial costs that are unlikely to have a payback. Were I a ratepayer in Auckland I might support it anyway, as a taxpayer outside Auckland I'm less likely to support it.

    ReplyDelete
  2. Cool bro, lets have cycle lanes some time after we get Marshlands road a four lane highway I see PaulL brilliant commenter below how can you do better Eric me and PaulL, but he an Australian, not take to much notice.

    ReplyDelete
  3. Oh, no disagreement with any of that. But all of it's going to be so second order compared to discounting vs not discounting a 40-year benefits stream! You'd knock 70% off of the benefits just by running PV on it! Doing the line-item analysis on the rest seemed kinda pointless by comparison.

    ReplyDelete
  4. Well, if discounting properly knocks off 70% of the benefits, and if 70% of the benefits are health-related, then actually only 30% of 30% of them are real. 20*0.3*0.3=1.8. So potentially the real cost:benefit here is 1:1.8.

    ReplyDelete
  5. When 70% of a purported benefit figure is demonstrably fabrication, I tend to assume that the rest is too. It's usually a safe assumption, and doing the line-by-line vetting takes a month.

    ReplyDelete
  6. Fair enough. I wonder whether our new fact checking organisation would be prepared to do that line by line analysis.

    ReplyDelete
  7. well I also tracked down the paper and I was hoping I could get some of the cashflow runs and make some estimates of the effects of discounting. As I skimmed the paper I had a faint hope that I might also find the outcome distributions from the monte carlo runs. But they must be in the supplementary material. Does anyone know where that is hiding? If so can you point me towards it.

    ReplyDelete
  8. It's at the journal website, here. http://ehp.niehs.nih.gov/1307250/ . Lots of sensitivity analyses, except for sensitivity to choice of discount rates.

    ReplyDelete
  9. Mostly amusement. :-)

    It kind of annoys me when people release studies like this. I would agree that there is likely to be a health benefit from more people cycling, and that it's in general a good thing. But when you use a faux economic analysis to attempt to show that it has a great cost benefit ratio, instead of just saying "we think it would be great and it would improve people's lifestyle", I feel like it's very "rent-an-opinion" or "rent-an-economist". Most people intuitively can tell that it's crap, and they start to assign that same rating to real economic analysis about things like minimum wages. Then we end up with stupid economic policies that all economists know are stupid.

    ReplyDelete
  10. There is no academic economist in New Zealand more sympathetic to your perspective than I am. 100% vehement agreement.


    The problem, though, is that there is just so much BS out there. To avoid that "debunking crap" take up like 500% of my time, have to triage and focus on the ones most likely to do substantial policy harm. We'd need an army of debunkers to keep up: it takes more time to reverse-engineer crappy studies than it takes the authors to write them!


    That said: if you do up the line-by-line analysis, I'm more than happy to point to it from here and cheer you for it.

    ReplyDelete
  11. It's definitely not my specialty area, but I may give it a go if I get some time. I'll let you know.

    ReplyDelete
  12. This will be a dumb comment in a sea of very intelligent ones: where is the best place to learn about discounts as it relates to economics?

    ReplyDelete
  13. Treasury's guide to cost-benefit analysis has lots of nice stuff, including discussion of discount rates. http://www.treasury.govt.nz/publications/guidance/planning/costbenefitanalysis/primer

    ReplyDelete
  14. Thanks Eric. Will look through tonight, been at WOMAD all weekend so no time for reading on the web :)

    ReplyDelete
  15. Well, I guess I was hoping for too much, though that there might be a spreadsheet or two, but no. Will take me awhile to process what the SM actually has in it, as on my first skim it seemed almost unintelligible. If I had ever prepared a business/investment case that looked like this I would have been laughed out of the room.

    ReplyDelete
  16. This is going to be an incomplete answer and I can't promise to get fully immersed in a lengthy debate. Short answer: we used the figure in the study - which is peer reviewed, published and was the subject of a PhD thesis. It's a big research project, we could not replicate it with our limited resources. They made the decision not to discount (as I understand it) because of many of the benefits are health and longevity related, and arguably society values health in the future as much as they value health today. This is a debatable point (that most people are unlikely to be very interested in), and while not intending to take a specific position on it, we went ahead used the study because it was highly relevant to our proposal. Longer answer: Benefit cost analysis is in theory used to compare and prioritise projects (though it is not at the moment in NZ). I accept that it may not seem fair to compare the BCR of motorway projects that have used a discount rate with this number which has not discounted benefits. However, the vast majority of the benefits (about 80%) in the motorway BCRs are "time travel savings" (which incidentally are unlikely to measure anything in reality. See Metz, Myth of Time Travel Savings). Let's be generous and assume time travel savings did exist; one could argue that a high discount rate is more likely to apply (you value a savings of 5 minutes today more than in 20 years time) than to health benefits. Moreover, motorway evaluations do not take into account sufficiently the impact of induced traffic, which erodes time travel savings quickly in peri-urban areas. So although that is not the purpose of discounting the benefits, I think it probably gives us a more realistic picture of the relative benefits/impact of the project over time. It's hard to see how walking/cycling infrastructure benefits would decline over time in the same way. Having studied at length the economic evaluation manual, I can say economic evaluation of road transport projects is highly incomplete (and therefore inaccurate) in NZ, and having looked at alternative methods of evaluation, I think it's quite safe to say that the cost effectiveness of walking and cycling for short trips is easily in the magnitude of 20 times better value for money than a project that introduces new road capacity (if not more). The straight up public infrastructure cost required to move one person one kilometer in a car is at least 8 times greater than by bicycle, but that doesn't include the private ownership and running costs of the car, and the parking, which would double that again. We haven't even started talking about externalities like congestion, noise, pollution, crash risk and public health. Finally, I do have a lot of questions about the appropriateness of high discount rates, and exponential discount rates. 1) The impact of high discount rates is to bias public investment projects towards those with shorter term benefits, this is a fundamental problem for sustainability. Obviously the Greens are a party concerned with the long term. 2) It's not clear that exponential discount rates accurately reflect human behaviour.

    I don't feel I've even begun to get into the logic and evidence to back up all these musings, it probably could itself be a good thesis topic.

    ReplyDelete
  17. I'll hoist this up into its own post and respond there. Thanks!

    ReplyDelete
  18. More general books on corporate finance:

    Principals of Corporate Finance - Brealey and Myers

    Valuation: Measuring and Managing the Value of Companies - McKinsey and Company

    Best website/blog - Aswath Damodaran Stern School of Business NYU

    http://pages.stern.nyu.edu/~adamodar/



    So much information etc on his pages that you will be swamped!

    ReplyDelete