Showing posts with label cost-benefit analysis. Show all posts
Showing posts with label cost-benefit analysis. Show all posts

Wednesday, 24 April 2024

Afternoon roundup

A closing of some of the tabs

First, a set from closing a pile of the week's accumulated stories from the Stuff papers. I wonder whether the people who complain about the absence of real journalism bother reading what The Post and Sunday Star Times have been putting out lately. 

And the rest of the tabs. Or some of the rest. I'm drowning here but the computer needs to be rebooted.... 

Wednesday, 10 April 2024

Afternoon roundup

The afternoon's worthies, as I close the tabs from one of the many open browsers....

Sunday, 20 November 2022

Transport Maths

Transport projects run their own version of benefit-cost accounting. It's weird to the sector, but apparently common internationally. 

Normally you'd want to go ahead with a project if it provides net benefits, where benefits are counted comprehensively and costs are counted comprehensively. If a project provides net benefits, it'll also have a benefit-to-cost ratio that's greater than 1. 

I never much worried about it whether they were using BCRs or net benefits.

But transport BCRs aren't really benefit-to-cost ratios

They're instead something a lot more like a "net benefit per dollar of transport expenditure" measure. Costs that aren't financial costs to NZTA wind up as disbenefits that are netted from benefits in the numerator of their ratio. 

And that difference can matter.

Imagine two projects, each of which would result in one statistical life-saving valued at $5m. That’s on the benefit side. Neither project has any other benefits. 

Project A would impose $4m in costs on drivers through reduced speed limits (increased travel time) and a $100,000 financial cost to NZTA in changing speed limit signs. So it has net benefits of $5m - $4m - $0.1m = $900k. 

Its true benefit-cost ratio, where all benefits are counted on the benefit side and all costs are counted on the cost side, is 1.2:1. NZTA’s 'net benefit per dollar spent' measure would have it as 10:1. 

Project B would spend $1m on median barriers and impose $100,000 through visual disamenity costs and hassles while the barriers are being installed. It has net benefits of $5m - $1m - $0.1m = $3.9m. The true benefit-to-cost ratio, where all costs are weighed against all benefits, is 4.5:1. NZTA’s 'net benefit per dollar spent' measure would have it as 4.9:1. 

In this example, project B has much higher net benefits, and a much higher true BCR. 

But a transport ranking would prefer project A. 

It's a made-up example with made-up numbers, but I wonder how many real-world cases wind up with this problem. 

If NZTA were mainly weighing up projects that had comparable bundles of financial and 'disbenefit' costs it probably wouldn't much affect things. But where more of the options, like blanket reductions in road speeds, mainly have costs that get netted from numerators while having trivial financial cost to NZTA, it could be a problem. 

It might be worth NZTA checking whether their method remains fit for purpose or whether it's likely to cause issues. Alternatively, it would be a fun student project.

Friday, 21 October 2022

Sense of scale

You'd think a half-decent test for policy would just be to stop and ask whether anybody would be willing to take the benefit of the policy, rather than just the per capita cash equivalent.

RNZ reports:

The government has revealed major changes to the way people will pay for public transport across the country.

Minister of Transport Michael Wood announced a $1.3 billion public transport single payment system in Auckland today.

The signing of the National Ticketing Solution contract with supplier Cubic means New Zealanders will soon be able to use a single payment system across all public transport networks.

People will be able to pay for bus, train and ferry trips using contactless credit or debit cards, Apple Pay and Google Pay.

New Zealand has 5.1 million people. Split $1.3 billion 5.1 million ways and you've got about $250. 

Is there anybody who would rather have the ability to use the same transit card anywhere in New Zealand (as well as Google Play and stuff), rather than $250 and just continuing to use Snapper or whatever in the place where they live, and throw a few bucks on a throwaway transit card when traveling?

Really?

 

Tuesday, 11 October 2022

Afternoon roundup

The afternoon's worthies:

Wednesday, 13 April 2022

Another occupational licensing cartel

Before the last election, Shane Jones set a new regulatory regime around forestry. It's now coming into effect. I'd forgotten about it.

Roger Partridge wrote about it back in 2020. It was a disaster both in content and in process. 

However, Jones has decided the industry can't fend for itself and needs an extra dose of regulation. Yet his actions suggest the industry's interests are not his main priority. His innocuous sounding Forests (Regulation of Log Traders and Forestry Advisers) Amendment Bill will sacrifice the interests of forest owners – among them North Island iwi owners – in favour of utopian plans for an enhanced domestic wood processing sector, which Jones hopes will boost employment, especially in his homeland in the Far North.

The Bill was introduced into Parliament under urgency on May 14. Indeed, Jones was in such a hurry that he gave the industry only a week to provide submissions to the Select Committee considering the Bill. (And for reasons best known to the Minister, the Environment Select Committee has been tasked with evaluating the Bill, not the better qualified Primary Production Select Committee.)

The Ministry for Primary Industry's Regulatory Impact Statement on the Bill was only made available to submitters two days before the date of submissions closed. This haste would be unacceptable in an advanced democracy like New Zealand, even if all the Bill did was set out an occupational licensing regime to ensure log traders and forestry advisers are "fit and proper people" – which is the Bill's ostensible purpose.

...

The RIS reveals serious shortcomings even with the Bill's more modest occupational licensing objective. The evidence relied on to support the proposed occupational licensing regime is described in the RIS as "qualitative," but it is clearly only anecdotal. Indeed, the RIS acknowledges shortcomings in the evidence available to define the magnitude of the problem faced by small forest owners arising from "time constraints on the policy development process."

This would be concerning enough if all the Bill proposed was an occupational licensing regime. It is in no one's interest for a regulatory regime to be introduced without a proper evidence base.

But the bigger problem lies with the more ambitious, unevaluated regulatory provisions of the Bill supporting the Minister's lofty plans for promoting "an enhanced domestic wood processing sector." The Minister's failure to subject these provisions to proper assessment – or to even raise them in his Cabinet Paper – shows a concerning disregard for due process.

Despite the lack of any decent evidence that an occupational licensing regime is needed for log trading, we're getting a new occupational licensing regime around log trading.   

This morning's inbox brings a reminder that this mess is now coming into force. 

In August 2020, Parliament enacted the Forests (Log Traders and Forestry Advisers) Amendment Act.  The Amendment Act (which was grandfathered under the Forests Act 1949 – presumably to speed its implementation pathway) provides a new regime for log traders and forestry advisers to be registered and subject to certain professional standards.

That new regime is to operate from 6 August 2022.

From 6 August 2022, there is a one-year transition period for log traders and forestry advisers to get registered before penalties will apply.

Public consultation on the proposed registration system closed in January and MPI is currently working on (and seeking feedback about) the new registration system.

The new regime introduces a professional compliance (aka occupational licensing) regime on a previously unregulated part of the forestry sector. 

The regime came about as a sort of kneejerk reaction from public discussion about the high proportion of raw logs which were being directly exported – and the concern that this was leaving a lower proportion of logs available for processing in New Zealand sawmills and/or impacting on the domestic lumber price.

This background is reflected in the stated purpose of the Amendment Act:

 

The Amendment Act seeks to achieve this purpose by introducing an occupational licensing and compliance regime for log traders and advisors operating in the forestry sector.  This seems to be despite the fact that the captured activities operate in a commercial or business context – and not at a consumer or retail level.  Amongst other things this seems rather puzzling - and arises the concern that, perhaps rather like the RMA, it will be used as a sword and not a shield by those (in trade) seeking some sort of commercial advantage over their rivals.

Captured activities – log traders

The Amendment Act seeks to capture the activities of ‘log traders’, being a person in trade who:

buys New Zealand logs, whether after harvest or in the form of trees to be harvested at an agreed time, and whether or not the person intends to on-sell the logs; or

exports New Zealand logs; or

processes New Zealand logs that the person has grown themselves,

and includes a person who does any of these things as the agent for another person:

(Also captured is a company that, in trade, transfers the ownership of New Zealand logs to or from a related company, whether the transfer relates to logs after harvest or in the form of trees to be harvested at an agreed time).

There is a prescribed volume threshold of 2,000m3 of New Zealand logs per year.  

Businesses involved in shipping and logistics are not caught by the Amendment Act.

Forestry adviser service

The Amendment Act also captures ‘forestry advisor services’. 

This is defined as someone who, in the ordinary course of business, provides advice on:

the establishment, management, or protection of a forest;

the management or protection of land used, or intended to be used, for any purpose in connection with a forest or proposed forest;

the appraisal, harvest, sale, or utilisation of timber or other forest produce;

the appraisal of a forest, forest land, or other forestry sector assets;

the application of the emissions trading scheme to forestry activities (within the meaning of the Climate Change Response Act 2002); or

the beneficial effects of forests (including, for example, how they contribute to environmental and economic outcomes).

(Also captured are those acting on behalf of others in relation to their sale or purchase of timber or other forest produce).

However, the Amendment Act does not cover the provision of advice, in a professional capacity, by those professional advisers who are already regulated by another professional body (e.g. real estate agents, financial advisors, lawyers and accountants).

Compliance and reporting standards

Those who are required to register as:

a log trader; or

a forestry advisor,

will need to satisfy the Forestry Authority (under the umbrella of MPI) that they meet the ‘fit and proper person’ standard. 

They may also have to provide evidence of their qualifications and experience in the forestry and wood processing sector – (the details of this requirement is s till being developed by MPI).

Once registered, log traders and forestry advisors will be subject to ongoing compliance, reporting and record keeping obligations.

A disputes (complaints) body is also established under the Amendment Act to hear and administer complaints.

Standard setting

The Forestry Authority has the power to set standards for any matter relating to forestry operations and delivery of forestry advisor services including:

land preparation, planting, forest management, harvest planning and site preparation and valuation;

biosecurity, sustainable land use, biodiversity and emissions trading;

sale and purchase agreements for domestic transactions or exports; and

other sale and purchase requirements.

However, the rule-making power must not impose any condition or requirement which would be a matter for commercial agreement between parties.

Code of ethics

The Forestry Authority may make rules that set a code of ethics for registered forestry advisers.

The code of ethics may include matters relating to:

professional responsibility (maintaining the highest standards of integrity and technical accuracy); and

responsibility to clients (including issues of confidence and conflicts of interest); and

professional work standards by registered forestry advisers in employment; and

maintaining professional competency.

Penalties

Penalties for non-compliance with the occupational licensing and compliance regimes established under the Amendment Act provided for fines of up to $40,000 for individuals and up to $100,000 for companies.

Timeline

Te Uru Rākau – New Zealand Forest Service (part of MPI) is working on a registration system which is due to come into force on 6 August 2022. Under the Amendment Act, log traders and forestry advisers must register to operate from this date.

That is, from 6 August 2022, there is a one-year transition period for log traders and forestry advisers to get registered before penalties will apply. 

I can understand why the Commerce Act exempts statutory regimes from Commerce Commission cartel enforcement. In some perfect world, there'd have been a weighing up of the costs and benefits of the occupational licensing regime through the legislative process. A CBA in the regulatory impact analysis would have shown that the benefits of the regime exceed the costs imposed by the restraint on competition. 

But that's not the Parliament we've got. The Parliament we've got passes these messes under urgency with rush-job regulatory impact assessments that can't possibly have tried weighing purported benefits against the cost of creating cartels. 

Just think about the range of people who could get swept up in the "Forestry Advisory Services" category.

Friday, 7 May 2021

The shovels weren't shovel-ready

There's opportunity here.
Less than half of the Government’s ‘’shovel-ready’’ infrastructure projects have begun by its first self-imposed deadline, with just 44 per cent of the 150 projects under construction by the end of February.
These things were set up as stimulus when everyone was worried about double-digit unemployment. Unemployment rates instead are below 5%. The projects never received any adequate CBA; the Infrastructure Commission Infrastructure Industry Reference Group just threw together a list of things that they might be able to get out the door in a hurry.

I was kinda sceptical that these things could wind up being delivered in a hurry; here's what I'd written on them a year ago

Why not just pause to reconsider all the ones that haven't started yet? Maybe they make sense, or maybe the money is better spent elsewhere. The government's announced a public sector pay freeze. Maybe they wouldn't have to do that if they could step back and reconsider the couple of billion dollars that they're here spending. 

The urgency behind the projects is gone. The government is short of funds. Why not take a minute to figure out whether the money could be better spent?

Tuesday, 19 January 2021

JEEM

Getting to Browser Tab Zero so I can reboot the computer is awfully hard when the one open tab is a Table of Contents for the Journal of Environmental Economics and Management, and every issue has more stuff I want to read.

A few highlights:

Monday, 24 August 2020

Covid and co-morbidities

A tallying of the costs of road accidents that included fatalities but ignored disabilities would result in too little investment in road improvements that might reduce accident rates.

Tallying Covid's morbidity costs is an awful lot harder than tallying the morbidity costs of road accidents. We have years of data on road accidents and anything in New Zealand that comes consequent to a road accident runs through our ACC system - it's then not all that hard to get a handle on costs.

Covid is a lot harder. We all hear horror stories about the ongoing consequences for some who catch it. If those stories represent one in a million cases, they wouldn't have much effect on policy decisions. If they represent one in ten cases, then they're a big deal. But it is darned hard to find anything that summarises the numbers.

Dr Jin Russell points to an article published in Nature back in July that summarises the various conditions that can be consequent to Covid. 

Here's the abstract:

Although COVID-19 is most well known for causing substantial respiratory pathology, it can also result in several extrapulmonary manifestations. These conditions include thrombotic complications, myocardial dysfunction and arrhythmia, acute coronary syndromes, acute kidney injury, gastrointestinal symptoms, hepatocellular injury, hyperglycemia and ketosis, neurologic illnesses, ocular symptoms, and dermatologic complications. Given that ACE2, the entry receptor for the causative coronavirus SARS-CoV-2, is expressed in multiple extrapulmonary tissues, direct viral tissue damage is a plausible mechanism of injury. In addition, endothelial damage and thromboinflammation, dysregulation of immune responses, and maladaptation of ACE2-related pathways might all contribute to these extrapulmonary manifestations of COVID-19. Here we review the extrapulmonary organ-specific pathophysiology, presentations and management considerations for patients with COVID-19 to aid clinicians and scientists in recognizing and monitoring the spectrum of manifestations, and in developing research priorities and therapeutic strategies for all organ systems involved.

The piece is written for clinicians, telling them things potentially to watch for. Turning it into something that could be used in cost-benefit assessment would be a pretty big job.  

There are rather a few potential consequences.



The article will note things like "in a cohort of 107 patients admitted to a single-center ICU with COVID-19, their rates of pulmonary emboli were notably higher than those of patients admitted to the same ICU during the same interval in 2019 (20.6% versus 6.1%, respectively)."

But using things like that in any CBA work would require knowing what proportion of COVID cases in that area wind up in ICU. You need to know what proportion of infections wind up with that outcome to be able to say anything . You can make simplifying assumptions about the proportion of cases that wind up in ICU. 

Other conditions are reported as fractions of hospitalised patients: 17% of hospitalised patients in Wuhan wound up with cardiac arrhythmias. But what fraction of infections wound up in hospital? And were those conditions presenting only while in hospital during infection, or did they persist? Acute kidney injuries occurred in 37% of patients hospitalised in New York, with 14% requiring dialysis. But again, what fraction wind up in hospital, and is this something from which people recover, or does it continue?

I really wish we had a better picture of the state of play.

There are a bit over 1500 people in New Zealand who have recovered from Covid. They would not be representative of the population overall, should New Zealand ever have a broader infection. Those here who have recovered are going to reflect the cases we have had: mostly those who have returned from overseas, who are younger and healthier. 

But despite that, surely it would be worth knowing the outcomes for that cohort. It would be cheap in the grand scheme of things. The government could pay GPs for a series of follow-up visits with those who have recovered, and pay those who have recovered for their time. It feels like it would be hard to spend $10 million on this. $5k split between GP and patient would only get you to $7.5m. 

The series of follow-up visits would check for any ongoing health effects of Covid. I'd also really like to get some measures on just how bad the whole experience was for those who have had it. I'm not a big fan of numbers expressed in survey values as compared to revealed preference, but I'd expect we'd learn something useful out of it here. 

Get the big table of conditions that have existing DALY or QALY figures attached to them. Ask each recovered patient "So, which would be worse? Getting Covid again exactly as you had it, but with no longer term consequences, or breaking your femur?" Running that over a series of conditions would get a distribution of valuations, among the cohort here who have recovered from it. You'd have to be rather careful in extrapolating that up to the population, because that cohort will have far fewer pre-existing conditions that worsen things. But at least it would be something of a start. 

I don't get why this kind of follow-up work isn't being done. Somebody would have to fund it, and it would likely be more expensive than a standard grant. Could it just be that nobody thought to put in a funding line for this kind of thing?

Thursday, 27 September 2018

MBIE discount rates

Treasury's cost-benefit guidelines say that the public sector should normally use a 6% discount rate in assessing normal projects.

People can argue about whether that's the right discount rate, and especially when it comes to very long term projects.

But at least having a single recommended discount rate puts all projects on a common basis. If it's weighted too heavily towards the present, it's so-weighted across all policy areas. No pick-and-choose to put in a high discount rate for projects with back-loaded costs to get it over the line, or a low discount rate for projects with front-loaded costs and later benefits.

MBIE's RIS on the government's Taranaki oil ban uses a different set of discount rates.

First up, we should be giving bouquets to MBIE for putting through advice that it had to know would be unwelcome. The government had already announced the ban - it simply didn't care about normal policy processes or potential costs. MBIE would have felt some pressure to say nice things about an already announced policy, and they didn't do that.

They have taken stick for the choice of discount rates though. They presented a range of costs to the Crown under different scenarios, including a 3% discount rate, a 10% discount rate, and an undiscounted flow of tax, royalties, and industry profits. Their midpoint estimates came from the 3% discount rate figures.

It looks like there's prescription in the rules around assessing permit applications requiring a 3% discount rate normally, and a 10% discount rate when proxying for industry cost of capital.

I don't know why that prescription is there. I am also not nearly a good enough interpretation guy to know whether this requires that rate in this context, or whether these rules were superseded by other rules I don't know about.

But I am curious why a 3% rate is here prescribed. I'm happy to entertain arguments that we should use a lower discount rate for really long term projects, but I'd want that to be set across all really long term projects rather than just some of them.

If anybody knows the backstory here, the comments section is open...

Thursday, 26 July 2018

Risks we don't even know about

Isn't it great that MBIE's out there, protecting us from risks that we don't even know about?




If only someone were out there protecting us against the risk of MBIE safety campaigns that manifestly fail cost-benefit assessment.

Treasury used to do that job, but since they kinda stopped hiring economists, they have had to save their trained economists for deemed-more-important work, like inventing wholecloth entirely new budgetary frameworks to account for living standards.

Our report on the scaffolding regulations is here; NZIER's confirmation that things were at least as bad as we thought is here.

And my critique of Treasury's hiring practices is in tomorrow's NBR.

Tuesday, 5 June 2018

Afternoon roundup

Today's worthies:
  • Cabinet has not yet produced a cabinet paper on the Taranaki oil ban, and Simon Bridges says that the government instructed officials not to provide advice on the ban. Even if you think that 'doing something about climate change' was part of a Labour/Green political mandate, wouldn't it make sense to make sure that whatever is done is the thing that can most cost-effectively abate emissions? If Bridges is right that the government instructed officials not to provide advice, can there be any good reason for that instruction? The most obvious explanations are not good. 

  • Kiwisaver provider Simplicity runs a very low fees model that is very attractive. But not one that's attractive to me, since they seem to have very strong non-return preferences baked into their model. If tobacco, gambling, oil or porn stocks started looking like attractive investment options, would they change their mind about the ban? 

  • The government's looking to repeal the three-strikes legislation. Farrar points out that three-strikes policy is fairly popular, but I'd be surprised whether people remember come 2020 unless crime figures become salient. I rather liked New Zealand's legislation, and especially in comparison to American examples. The point of three-strikes, from an economic perspective, is to maintain marginal deterrence. In short, you need a stronger expected formal penalty for a second offence or third offence than you do for a first offence to achieve the same deterrent effect. Why? Because the first offence comes with a giant informal "Now you have a criminal record and a whole pile of things you thought you could do with your life are now going to be very very hard" penalty. That informal penalty's sunk after you've got the first conviction, so you need a stronger formal penalty for the later offences. And where California induced problems by having the same harsh penalty for second and third strikes across broad classes of offences, New Zealand maintained proportionality by linking everything to the sentence-specific maximum penalty. But, all that said, I doubt there'll be any particular effect on crime. There were just too many high profile cases where judges thought any application of the strike penalties was unjust, and so invoked their discretion (in my view) inappropriately. If folks don't expect the penalty will be applied because the judges won't apply it, the law's useless even if it's great in theory. 

  • Is there any simpler explanation for the meth-mess than that Housing New Zealand had excess demand for houses and using an insanely sensitive hair-trigger for evictions let them free up some houses? Plus the usual stories around how agencies are more likely to be punished for not being sufficiently risk-averse than for being too risk-averse

  • And, finally, some good news. Catherine Healy is now Dame Catherine Healy. She heads the New Zealand Prostitutes' Collective and helped see prostitution legalised in 2003. And how can you not love a union that, on seeing abuses of migrants on temporary visas illegally working in the sex industry, argues for legalising their work too instead of having more labour inspectors going around to deport competitors? America's ahead of us on marijuana reform, but we're miles ahead on this one. Too many Honours go to career public servants whose main merit was having diligently undertaken their day-job for 40 years. This one isn't like that. 

  • David Friedman at Oxford Union on market failure. HT: Jim Rose.  

Tuesday, 22 May 2018

Reader mailbag: Caveats on policy costings

This morning's inbox came with excellent comment from an informed reader about the difficulties in getting reasonable independent policy costings. The email reads (and is shared with permission):

  • Do not underestimate how effective agencies can be at blocking information they do not want others to know. One reason the costings will be poor is because all kinds of information will not be unavailable because the agency, not the minister, do not want to be challenged. You will find "privacy" and "computer systems" becomes big issues...
  • The main advantage of a single separate agency is consistency in the assumptions. At the moment they change depending on the whim of the needs of a budget bid. If agencies have to write down and be long term accountable for the assumptions, that will be a big step forward in like for like comparison. Even if the costings themselves are no more accurate, they comparison between policy costs (bigger, smaller, etc) will be more robust
  • My experience of independent agencies is not good: stats NZ, Office of the Auditor General, Ministry for Women, SuPERU, etc They are all small agencies with some kind of "advocacy" role that they use to justify pursuing their own agenda. Ministerial accountability has its limits, but it is at least real accountability.

I agree with most of this. Privacy seems to be the default "Computer says no" reason for not doing things. Also agree that on the advantages of having a single consistent, if biased, ruler over having multiple different rules with different biases. 

My prior piece had looked at the ways a party might avoid having rigourous costings applied. Accountability for the costing agency would be a fun one - perhaps reinstate betting markets on whether policies wind up costing what they're advertised to cost? Ex-post assessments?

Wednesday, 11 April 2018

The Heckman Curve is flat

It sounded good in theory. Interventions targeted at youths could very plausibly have been rather more effective than programmes targeted at older cohorts. Heckman's foundation thing made a pretty infographic about it.

But it's fundamentally an empirical question. The infographic is based rather more on intuition than on any real lit survey. 

Rea and Burton checked it out. They went into the big Washington State intervention database, sorted interventions by age, and just plotted out the cost effectiveness. There was no curve - just a noisy mess. And the regressions found nothing either. 

Here are the key tables. 





Figure 3's the kicker. Compare it to the stylized Heckman curve. Ocular least squares really are enough here. 

It also links into my other worries about the Perry preschool stuff that Heckman's been pushing and that seems really not to have fared well in broader roll-outs in Tennessee or in Quebec. 

If there is a Heckman Curve, it's hard to see it in the programme evaluation work compiled in the Washington State database

Wednesday, 28 March 2018

The Economist's Cup

I had an awful lot of fun over at The Spinoff imagining having $212 million in public funding to support a world competition of economists - the Economist's Cup. And if you think there are great benefits to the country from putting that kind of money into the America's Cup, my proposal's even better.

A snippet:
I know you’re going to be sceptical about this but hear me out.

The 2021 Economist’s Cup should be held in Wellington. This annual event, which will first be held in three years, will bring some of the world’s best economists to Wellington to compete in being great at being an economist.

You might not think this is much of a spectator sport and that it might be a little too niche for the $212 million in public funding the event would require.

And you would be wrong.
If you have half as much fun reading it as I had writing it, you'll still have a lot of fun. 
 

Monday, 5 March 2018

For better costings

Parliament deserves better advice than it has been getting about the policies coming up under Labour's 100-day plan. There just isn't adequate time for the Ministries to produce reasonable regulatory impact assessments under those strictures. 

In this week's column over at Interest, I argue that where the Ministries haven't been able to produce a non-caveated RIS, the policy should undergo mandatory Post-Implementation Review. One reasonable way of doing that would be to sunset the legislation for six months after the due date for a mandatory PIR.

A snippet:
Imagine that you and your partner agreed that you would buy a house together after the wedding – and you both had your eyes on a particular property. After the celebrations, you hired a building inspector to check the place out.

Would it really be sufficient if the inspection report said only, “You promised that you would buy a house together after the wedding. This is indeed a house. You must purchase it.”?

I’d expect any of us would refuse to pay the builder’s invoice. But I also expect that we would give the builder more than five minutes to inspect the place.

I wish that this were about something as inconsequential (in the grand scheme of things) as one couple’s imaginary experience with a dodgy building inspector.

But some of the advice that Parliament has been receiving about the policies contained in Labour’s 100-day plan are every bit as inadequate as that builder’s report.

...

The bureaucracy is a servant of its political masters, and elections do matter. Newly elected governments have a mandate to implement their policy proposals. But in the absence of better official costing and evaluation of election policy platforms, Parliament often just does not have adequate information from the bureaus to assess whether policy promises still make sense after the election.

There is no simple solution, but we might consider one small improvement.

The Ministries will not always have time to provide an adequate assessment of the effects of policies. That can happen because of post-election pressures, but it can also happen because of unforeseen events between elections causing rapid political response.

When proposals have not had a chance to be more thoroughly evaluated, they should undergo more rigorous post-implementation review. A lot of Regulatory Impact Statements promise these reviews, but few seem ever to be undertaken. The reviews should be compulsory for policies undertaken under 100-day clocks.

When we buy houses, we make the purchase conditional on a good builder’s inspection. There often isn’t time to get a proper inspection report before making an offer, so we build in room to get it done properly after the offer has been made. And if the inspection report turns up cracked foundations, we have an out. Should we really expect less than that when it comes to policy changes affecting the whole country?

Friday, 29 September 2017

In praise of evaluation

The Ministry of Social Development has been evaluating the effectiveness of its employment assistance programmes.

Here's the key figure.

Note that effectiveness is here defined by whether the programme improves participants' outcomes across income, employment, and independence from welfare. It does not look like it measures cost-effectiveness, but would be a first step toward that. They note that future editions will try for a Welfare Return on Investment measure, and a Social Return on Investment measure.

A big chunk of spending couldn't be evaluated because it's on the childcare assistance programme that's available by entitlement to anyone who asks, and has been for as long as they have reasonable data. And that programme is $183 million.

I wonder whether they might be able to get some mileage by exploiting shortages in childcare availability. After the ECE subsidies came in for more families in the mid-2000s, and the rule changes around qualified ECE staff, there were lots of stories about shortages of available childcare spaces. I know we had our application in for daycare at Canterbury University months and months ahead of Ira's being born. Anyway, if there is any data on where there were and were not shortages of childcare, then that might be exploited to evaluate the effectiveness of the childcare assistance programme. They could also consider the discontinuity at the income boundary for eligibility. 

Kudos for the evaluation work! I hope that this kind of work continues, and look forward to the cost-effectiveness versions to come. 

Monday, 1 February 2016

Costing policy

My Friday column at The NBR noted some of the potential difficulties in the Greens' proposed policy costing regime.

  • Housing it in Treasury is a bad idea as compared to in an independent office of Parliament, so it would not have to conduct a substantial body of work that it would need to keep secret from the Minister overseeing the agency;
  • A rule or norm that every policy be costed means that it will be more difficult to cost policies than you might expect for currently costed policies. Why? Right now, parties can refrain from costing silly policies. In the alternative, they'll have reason to delay and obfuscate, making it harder for agencies to undertake a costing;
  • New spending at each election can be big, but even bigger is the body of existing policy that is rarely revisited.
And so I concluded:
But the bulk of the government’s spending – the big line items that do not change much from budget to budget – receive rather less attention.
Take the government’s student loan programme. Labour’s zero-percent student loan policy drew attention during the 2005 election campaign and again in 2008. But we do not often hear discussion of the $5 billion gap between the nominal value of the loan and their carrying value.
The vast majority of government spending simply carries on, from year to year, without much notice.
Perhaps an agency established to cost new political party proposals could, between elections, spend its time running rolling reviews of existing spending programmes, ensuring they continue to deliver value for money.
After I filed the column on Wednesday, I found that Mike Reddell had some similar things to say. He also helpfully points to international examples. He also suggests it could be at least as good to simply provide funding to political parties that they then be able to commission independent evaluations. Here's Reddell:
On balance, I still think there is a role for something like a (macro oriented) fiscal council in New Zealand, perhaps subsumed within the sort of macroeconomic or monetary and economic council I suggested here (but perhaps that just reflects my macro background). And there is probably a role for better-resourcing select committees. But when it comes to political party proposals, if (and I don’t think the case is open and shut by any means) we are going to spend more public money on the process, I would probably prefer to provide a higher level of funding to parliamentary parties, to enable them to commission any independent evaluations or expertise they found useful, and then have the parties fight it out in the court of public opinion. The big choices societies face mostly aren’t technocratic in nature, and I’m not sure that the differences between whether individual proposals are properly costed or not is that important in the scheme of things (and perhaps less so than previously under MMP, where all promises are provisional, given that absolute parliamentary majorities are very rare). If there are serious doubts about the costings, let the politicians (and the experts each can marshall) contest the matter.

Saturday, 7 November 2015

Runway questions

Wellington City councillor David Lee is asking the right questions about the proposed Wellington Airport runway extension.

All proposals thus far have the airport, one-third owned by Wellington Council and two-thirds owned by Infratil, paying for about a third of the extension's costs; the remaining two hundred million would have to be covered by central government and councils in the region. Different numbers float around, but it would be in that ballpark. I'm not sure whether the funding would be provided as a straight subsidy or whether the funders would be taking an equity position.

Lee critiques the economic impact assessment that highlighted the substantial effects that a big increase in airport traffic could bring. But economic impact analyses are not cost-benefit assessments and pretty typically count things like total visitor expenditure rather than net profit. NZIER pointed out the numerous differences between CBA and economic-impact figures in their rather thorough critique earlier this year.

If the projections in increased flights are right, and there's still about a $200 million gap that can't be recovered through slotting fees on the increased long-haul traffic, then any CBA would need to show real net external benefits of the project in the order of $200 million. Council could be persuaded by ones where the benefits include diversion of traffic from other parts of the country; central government should be looking for real evidence of net benefits to the country as a whole after accounting for diversions from other airports.

It's entirely plausible to me that we'd see an increase in flights from the Singapore hub to Wellington with the airport extension; players that currently don't fly to New Zealand might tap in and wouldn't have been included in existing surveys. But is it plausible that we get $200 million in real net external benefits from the extension? It'll be interesting to see the cost-benefit assessment when it comes out.

Previously:

  • Optimal airports, in which I critiqued some of the benefits suggested for an extension.

Tuesday, 13 October 2015

Catastrophes are complicated

There are lots of potential catastrophes out there, each of which might be worth spending something to avoid. So how should we decide things in a world of multiple catastrophic scenarios? Martin and Pindyck show that things get messy quickly. [ungated]

You can't run a catastrophe-by-catastrophe cost-benefit assessment and just work your way down the list: as each has a big potential effect on current consumption (through taxes to provide mitigation) or on future consumption (if the catastrophe happens, or cumulative effects of the mitigation works), which catastrophes you want to avoid has to be decided simultaneously across all projects.

Here's perhaps an easier way to think about it. Your expensive home renovation works might pass cost-benefit for you. Upgrading the car might also pass cost-benefit. But if you do the renovations, the car upgrade will no longer make sense because you won't be able to afford to eat, and the same if you do the car upgrade before the renovations. So you need to look at them both at the same time. Worse, if part of the home renovations is to keep the garage roof from wrecking the car, then the car upgrade is pointless if you don't do the renovations.

How things map out then depends on both your time preference rate and on your risk aversion. Here's the results of some simulations over seven potential catastrophes, with their rough modelling of the effects.

With low values for time preference and for the risk aversion parameter, it makes sense to pay to avoid a pile of potential catastrophes. As those change, so too does optimal mitigation.

I note that they left out the catastrophe that worries me: asteroid impact.