Wednesday 19 March 2014

Genter on cost-benefit

The Greens' Julie Anne Genter provides some helpful comments on the cost-benefit assessment cited in their cycling proposal.
This is going to be an incomplete answer and I can't promise to get fully immersed in a lengthy debate. Short answer: we used the figure in the study - which is peer reviewed, published and was the subject of a PhD thesis. It's a big research project, we could not replicate it with our limited resources. They made the decision not to discount (as I understand it) because of many of the benefits are health and longevity related, and arguably society values health in the future as much as they value health today. This is a debatable point (that most people are unlikely to be very interested in), and while not intending to take a specific position on it, we went ahead used the study because it was highly relevant to our proposal.
I sure wouldn't have expected the Greens to come up with their own full C-B on this one. But it was pretty obvious that the paper's authors had decided to eschew discounting; that made the figures incomparable with other LTNZ projects. Further, plenty of other roading project cost-benefit analyses build in health and value-of-life benefits from reduced accidents which also accrue over a longer time period. We then can't really say that the policy would take money from lower-valued projects and move it into this higher-valued one, since the basis for evaluating value differed too much across them.
Longer answer: Benefit cost analysis is in theory used to compare and prioritise projects (though it is not at the moment in NZ). I accept that it may not seem fair to compare the BCR of motorway projects that have used a discount rate with this number which has not discounted benefits. However, the vast majority of the benefits (about 80%) in the motorway BCRs are "time travel savings" (which incidentally are unlikely to measure anything in reality. See Metz, Myth of Time Travel Savings). Let's be generous and assume time travel savings did exist; one could argue that a high discount rate is more likely to apply (you value a savings of 5 minutes today more than in 20 years time) than to health benefits. Moreover, motorway evaluations do not take into account sufficiently the impact of induced traffic, which erodes time travel savings quickly in peri-urban areas. So although that is not the purpose of discounting the benefits, I think it probably gives us a more realistic picture of the relative benefits/impact of the project over time. It's hard to see how walking/cycling infrastructure benefits would decline over time in the same way.
I agree with much of this. But surely then the correct answer is to run it using standard time discounting, get a smaller but defensible number, and then point out that the benefits of other projects are overestimated. It wouldn't surprise me if the benefits for other roading projects were overstated. And even if the cost-benefit analysis were right, there are plenty of relatively low benefit-to-cost ratio projects out there; NZTA put the whole of the Roads of National Significance programme at a 1.8:1 benefit-to-cost ratio. I wouldn't be surprised if a sound version of the cycling benefit-cost study beat 1.8. Why say 20:1 when you only need to beat 1.8:1?
Having studied at length the economic evaluation manual, I can say economic evaluation of road transport projects is highly incomplete (and therefore inaccurate) in NZ, and having looked at alternative methods of evaluation, I think it's quite safe to say that the cost effectiveness of walking and cycling for short trips is easily in the magnitude of 20 times better value for money than a project that introduces new road capacity (if not more). The straight up public infrastructure cost required to move one person one kilometer in a car is at least 8 times greater than by bicycle, but that doesn't include the private ownership and running costs of the car, and the parking, which would double that again. We haven't even started talking about externalities like congestion, noise, pollution, crash risk and public health.
Julie Anne knows more about transport than I do, and I believe her to be a truth-seeker. I will make a couple of points here though.

It's entirely possible for it to simultaneously be true that the public infrastructure cost of roading is higher than for cycling, and for it to make sense to invest in roading.

Petrol excise taxes pay the majority of the bill for national roading projects and half the bill for local roading investments, though I'd assume that Genter and I would both agree that we shouldn't be spending out of general revenues for roading projects over and above that which is supported by excise and road user charging.

For some people, cycling's a great way of commuting. For others, not so much. Our family carpools together across town every day. Cycling wouldn't work well for us. I'm happy to pay more in terms of petrol (including excise), car maintenance, parking, and everything else, than to cycle that route every day. And especially since we get random-draw unpredicted rain. If you asked me, "Hey, Eric, how much would we have to pay you to use a bicycle for your commute every day with the kids," I would tell you that you would have to pay me about $30,000 per year to make me consider it. It would substantially reduce my quality of life. I imagine the horror of getting a call from daycare that the 3 year old had come down with a stomach flu, was vomiting, and that I'd have to take her bicycle. With the 6 year old tailing along behind on his bicycle because there's no way I'd have a 6 year old cycle from Ilam to Brighton on his own. Because of all that, I'm real happy to pay the costs of driving a car rather than cycling. Further, the time in the car isn't pure loss either: we play games with the kids (traffic permitting) or listen to audiobooks - we're half-way through Robinson Crusoe currently. They'll be getting much of the Canon courtesy of the commute. That counts for something.

Let's consider how the model the cited paper uses handles this problem. Think about the person currently on the margin between cycling and driving who currently drives. She's weighed up all of the costs of driving, all of the health benefits of cycling, and just barely prefers driving. Now suppose we made cycling easier and, because of where she lives, she now just barely favours cycling. The benefits to her of the programme should really be pretty close to zero: she just barely preferred driving before, and just barely prefers cycling now. But the simulation model would count as benefits to her all of the net reduction in health costs (reduced driving accident risk, plus bicycle accident risk, minus health benefits from exercise) and all of her reduced fuel expenditures, without counting any of the costs of the switch as she perceives it. No costs of cycling, no inconvenience in the rain, no hassles in trying to get groceries home in a mixed-purpose trip, no increased food expenditures to fuel the cycling, no time costs for her longer cycling commute.
Finally, I do have a lot of questions about the appropriateness of high discount rates, and exponential discount rates. 1) The impact of high discount rates is to bias public investment projects towards those with shorter term benefits, this is a fundamental problem for sustainability. Obviously the Greens are a party concerned with the long term. 2) It's not clear that exponential discount rates accurately reflect human behaviour.

I don't feel I've even begun to get into the logic and evidence to back up all these musings, it probably could itself be a good thesis topic.
Ok. Here's the cost and benefit time profile on the Roads of National Significance project.
Suppose that all the real benefits are only half of what's here presented, just for sake of argument. Even with that, what do you suppose that a zero discount rate would do for the cost-benefit assessment of RONS? The benefits from the RONS programme are heavily backloaded while the costs are frontloaded. Time discounting doesn't just affect bicycling projects with later health benefits. 

And the tabulated RONS benefits aren't just commuting time savings: they've got reduced congestion (and associated costs), accident reductions (health benefits), and vehicle operating costs savings. The green curve adds in a bunch of wider economic effects. They'd reckoned on $690m per annum in non-market benefits like lives saved; it isn't all "saving 5 minutes a day in RONS versus saving lives in cycling projects."

I'm not going to go to bat defending RONS or the cost-benefit study on it: I know there's been plenty of criticism of it. But if the general point is that time-discounting isn't fair for projects like cycling infrastruture that have lots of potential future health benefits, well, the "front-loaded costs, later benefits" profile also fits RONS. 

Again: it seems completely plausible to me that, if government is going to be devoting general revenues to roading projects rather than just petrol excise revenues, it could well make sense for central government to use some of that funding to top up local government expenditures on cycle paths instead. But my confidence in the proposition is reduced rather than enhanced by that Auckland study.

Finally, I worry a lot about differential cost-benefit methods being used across different kinds of projects. Suppose Genter's right and that health benefits should attract a zero discount rate rather than a standard one. We're currently applying standard discount rates on every other project out there: investments in new hospitals, various public health interventions, pretty much everything. We don't get formal cost-benefit analyses on every policy, but big ones do require an RIS that has to at least have indications of net present value. And the RIA Handbook under which the Ministries are supposed to be producing these things explicitly cites the Treasury cost-benefit analysis primer as guidance. If we're going to discount health benefits at zero, it's a dumb idea to apply it piecemeal. Get it into the Treasury guidelines. Otherwise, you risk massively privileging those health projects that get the zero discount rate over ones that don't. 

I'd argue pretty strongly against a zero discount rate for health: it's pretty strongly contrary to observed behaviour. But if you're going to do it, it has to be an across-the-board decision put formally into the Treasury guidelines, not something pulled out to weight the scales for favoured projects. 


  1. Interesting post Eric (and correspondence). I still can't bring myself to try and make sense of the paper's SM and attempt to actually do some discounting, plus my first pass through made me wonder if all of the actual results were in the paper and SM.

    Anyway, the discussion on discounting is interesting, as is it relates to its general application and long life assets. I think the biggest weakness in many arguments about what the "correct" discount rate should be fail to consider that discount rates, CAPM etc are all approximations to what happens in reality. As such there is no "correct" discount rate.

    As you rightly point out you and I would value cycling differently, given our individual preferences for travel, time management etc. But, if we don't apply a rule set of some description then we can really just make up whatever we want.

    While I prefer to apply some analysis to determining the appropriate discount rate for an investment, I also from a personal perspective tend to think of the results of DCF analysis as providing relative value assessments, given a consistent analytical framework.

    I have seen too much debate where people don't like the outcomes because of personal preferences i.e. think roads are worse than cycleways. Debate the inputs, debate the analysis, but a consistent framework must be used or no valid judgements can be formed or relative preferences determined.

    My 2c.

  2. My economic literacy is minimal, so apologies if the answer to my question is blindingly obvious. But: given the example of your own transport requirements, which seem reasonable, is it possible to put a value on the benefit to you of *someone else* switching from favouring driving to favouring cycling? If a desired goal of you paying road user charges etc. is to increase the capacity and efficiency of roads on your own commute, then making it possible for others to avoid sharing the road with you could end up being the best use of that money.

  3. Great post, but I think Genter probably deserves more credit for taking the debate up in Parliament in an intelligent way with minimal financial resources. Like you I strongly suspect once all the numbers are in they are going to look pretty good for cycling investment and I hope the funds become available to do some proper C-B analysis.

  4. Its hard to listen to anything else after someone promotes something as blatantly false as a 20:1 benefit to cost ratio.

    How do you even remain civil enough to consider the validity of other points when the first is a flat out lie?

    I suppose someone can be a truth-seeker, without being a truth-promoter. But what use are they?

  5. Broadly I agree with that, but in countries like this one, that genie is well and truly out of the bottle. A better example, I think, than your STDs was the cost imposed on the rest of us (often into the millions) to rescue people who have deliberately put themselves at risk up tall mountains or in high seas. Never sure why we are paying for them. And I think something like a tobacco tax is an attempt to make the users, and those who profit, "pay an extra couple of grand" without missing out on the general benefits that come from a public health system.

  6. I agree entirely with you for mountains and the high seas. The big big difference between the cases is that, for sugar, they want to ban people from eating it or tax us for eating it, whether or not it results in harm; for mountaineering, they only charge you for the rescue if you wind up needing to be rescued. I don't need to be rescued from any of my current consumption.

  7. I agree it is not unreasonable for truth seekers to be wrong. My issue is with truth-seekers who are lie-tellers.

    Comparing discounted figures to non-discounted ones is BS. And I dont buy the defense that that lie is just an emphatic truth.

    20:1 benefits to costs? Lie. And the liar loses the right for the rest of their case to be heard. I trust honest people. If you are right, you usually dont need to lie.

  8. There seems to be an assumption here and in the discussion that a 20:1 BCR is clearly impossible for a transport project. Yet we achieve it quite regularly with low-cost road safety works, i.e. simple engineering treatments that greatly reduce the risk of deaths and serious injuries at blackspots (yes, I guess there's a whole other discussion about how accident savings are valued...). Some projects even have 100:1 BCRs.

    That aside, I was also interested in your theory of a current driver who assesses the benefits as marginally in favour of driving over cycling, but who might switch to being marginally in favour of cycling following some investment in that area. That does somewhat assume that the person has a rational understanding of all the costs and benefits of each mode; I'd argue that actually most people have a poor perception/understanding of the costs/benefits personally of cycling. Issues like travel time, safety, weather, hills, carrying stuff, etc that seem to matter a lot to non-cyclists actually matter less to people who regularly ride - they've realised that they're not as bad as they thought they were or worked out ways around them. So it might be that someone who's crunched their (perceived) numbers and determined that cycling is now a slightly better option than driving might actually be pleasantly surprised in due course about the benefits they are gaining by cycling.

  9. Food labelling and food taxes expert Dr Mike Rayner (based at
    Nuffield Department of Population Health at the University of Oxford)
    makes the point that poorly designed food taxes could have unintended
    negative consequences. The paper by Fletcher et al. examines the effects
    of soda taxes per se, which appear to me to be an example of a poorly
    designed tax. The soda taxes they examined were applied (and modelled)
    to fizzy drink but other sugary drinks were exempted. Unintended effects could
    include, for example, consumers switching to other non-taxed but caloric drinks and beverage manufacturers developing (and promoting) new non-soda beverages as a means minimising any effects on consumption and thus profit. Indeed the data presented suggested this occurred.

    I think such taxes need to be applied to all categories of
    sugar-sweetened beverages. Mike Rayner and colleagues predict (using
    econometric models) that a 20% tax on sugar drinks will reduce obesity prevalence by 1.4% in the UK. This, of course, remains to be proven. It will be of
    great interest to follow the Mexican experience with it’s newly
    implemented 10% tax on all sugar-sweetened beverages (i.e not just soda).

    It is also worth considering that during the early 90s (the period
    of time the Fletcher et al. analysis covers) intakes of free or added sugars in the
    US were at an all time high with the low fat obsession driving the
    development of of a huge range of low fat but highly sweetened
    convenience foods and drinks. I’m no econometrician but I wonder whether/how
    this could have confounded their analysis. Nevertheless
    this illustrates the importance of developing multi-pronged, well
    thought out policies for addressing excess calorie consumption and
    population obesity. Focusing all our efforts on simple fiscal measures,
    in isolation, could be quite pointless.

  10. Agree that if it's going to be done, it has to be across-the-board rather than specified products. And not just beverages: every single thing with sugar as an ingredient or component as you could reasonably expect substitution between drinks and other products. I disagree that it should be done, but making it comprehensive at least means it's not a dumb way of achieving an end with which I disagree.