90 Days

Motu's had a look at the effects of the 90 day trial legislation. Under that legislation, employers could hire employees on a trial basis and dismiss them relatively easily within that 90 day window. Supporters of it expected it to encourage employment of riskier employees; opponents expected substantial churn: that employers would somehow figure it made sense to hire people for three months, fire, rinse and repeat.

The Motu study, undertaken by Nathan Chappell and Isabella Sin, two fine Canterbury economics graduates, uses a beautiful little natural experiment. Firms smaller than 20 employees were allowed to use the provision; those over it were not. At least for a time. Afterwards, it extended. But you had a nice little period in which there was a discontinuity at 20 employees. They then looked at hiring data in for firms in the 15-25 employee range. If the bill had effects, that's where the difference would show up.

Using that experiment they find, well, very little in the aggregate. There was no particular boost to employment, but neither was there any churn. It didn't seem to do anything at all.

But there is a bit of a problem in focusing on the aggregate. If you're looking at effects across all firms, and only a minority of firms would ever want to use the trial periods, then if there were an effect for that group of firms, you likely wouldn't see it in the aggregate data. The data doesn't let them tell which firms actually elected to hire new employees on trial arrangements, and which offered permanent contracts from the get-go.

Or to put it another way, suppose that some medicine reduced your chance of death from a relatively uncommon disease by 10%. You wouldn't notice any effect at all in overall aggregate national death rates. But you would notice it if you looked in the places where it were used.

This could matter. They cite MBIE work showing greater uptake of trial periods in construction and wholesale trades and low use in education and training; they then find a about a 10% increase in hires among small firms in industries known to use trial periods who were eligible to use trial periods as compared to small firms in the same industry that were just a bit too big to use trial periods. Those small firms eligible for 90-day trial periods in high-use industries had about a 7% increase in long-term hires.

Evidence that it particularly encouraged employment of riskier employees is rather weak, or at least riskier as measured by things like being a former beneficiary.

On the whole, it looks like the policy provided an increase in employment in construction companies and wholesale trades, no increase in churn or dismissals. If you think that people bear substantial psychological costs of a 90-day trial period where actual dismissal rates are trivial, then you might not like the policy. If you think that those are likely to be minor relative to the employment benefits in the sectors that need it, then the policy remains a good one. And if you're going to weigh the psychological costs of uncertainty for employees under trial periods, weigh too the psychological costs for employers having to make hires under conditions where firing is very difficult.

It might not be as beneficial as we had hoped at the outset, and so it might have been a mistake when evaluated against other beneficial policies on which the government could have expended political capital (youth minimum wages, for example), but it's a policy well worth continuing.

Crossposted from The Sandpit.

Marsden Maths

Motu's released the working paper they'd presented at the NZAE meetings earlier this year. They show that Marsden grants do increase research output. But as for whether the programme is cost effective? Well, that's more fun.

The average researcher on teams receiving Marsden grants made 6 proposals and received 1.2 grants over the period 2000-2012. The Fund allocated just under $68 million in 2013; the standard grant's maximum budget is $300,000 per year while Fast Start grants are limited to $100,000 per year. Panels reject 71-84% of first round proposals. The Motu analysis looks at second round proposals, 41% of which were funded.

Results? Here's the big headline chart from their press release.

So about the biggest effect that they found was that a researcher who received two successful research grants received a 10% increase in publications and a 14% increase in citations. In the regression discontinuity design, comparing project teams whose proposals just made the cut with those that just didn't, they found no effect.

Their abstract's version:

Overall, we find that funding is associated with a 6-15% increase in publications and a 22-26% increase in citation-weighted papers for research teams. For individuals, funding is associated with a 3-5% increase in annual publications, and a 5-8% increase in citation-weighted papers for 5 years after grant; however, the lag structure and persistence of this effect post-grant is difficult to pin down. Surprisingly, we find no systematic evidence that the evaluation of proposals by the Marsden system is predictive of subsequent success. We conclude that the Marsden Fund is modestly successful in increasing scientific performance, but that the selection process does not appear to be effective in discriminating among second-round proposals in terms of their likely success.

An annual 8% increase in the number of citation-weighted papers for five years after a grant might amount to one full paper over the entire time period. A research grant costing upwards of a hundred thousand dollars per year seems a pretty costly way of generating one citation-weighted paper.

It would be ... interesting ... to compare the cost-effectiveness of Marsden with an alternative scenario in which academics who maintained a reasonable publication record received an unconditional grant and the money spent on Marsden-related administration went instead into hiring more academics. There have to be at least 30 full-time equivalent academic-years that go into Marsden grant writing and grant evaluation.

Insulation

I didn't know that the government commissioned work on the effectiveness of its Clean Heat programme which provided subsidised insulation. Arthur Grimes reports in the latest MOTU update that they matched treatment homes with a set of comparable control homes and ran difference-in-difference estimation on the effects. He writes:

The energy study showed that insulation treatment caused a statistically significant, but small (0.7%-1.0%) fall in metered energy consumption. The small drop in energy use is consistent with an economic model in which energy efficiencies were obtained from the insulation so that the effective price of heating fell, in turn resulting in increased consumption of heat (i.e. a warmer house). Greatest energy savings were experienced in cool areas. Measured energy use was shown to increase slightly with the installation of clean heat installation (no data were available on nonmetered energy use).

So free insulation will not help reduce energy demand: people respond to the reduced cost of heating by consuming more of it. This is worth knowing as some parties think that improved insulation is a substitute for greater generation capacity.

Health outcomes improved consequent to better heating. The MOTU reports found a 3.9:1 benefit-to-cost ratio, but that 71% of total benefits were from reduced mortality. As a public health intervention to reduce mortality, this could be fine. But it is hard to make a market failure case for the subsidy scheme.

Imagine two possible policies. Policy A gives cash to households and lets them choose whether to insulate their house with it; they're also given a pamphlet listing all of the benefits of insulation including increased life expectancy. Maybe it also has a nice narrative about how nice it is being in a warm house. There are lots of suggestions about how the money should be used for insulation, but it leaves the choice up to the household. Policy B gives a voucher for home insulation that can only be used for home insulation. If in the Policy A world households choose things other than insulation, can we really say that the the insulation subsidy programme passes cost-benefit analysis relative to the "give money to poorer households" programme? I don't think so.

Sure sure, there are other benefits where you could make a case: reduced hospitalisation and the like. But if 71% of the benefits were reduced mortality (ie reduced losses in VSL), then the benefit to cost ratio without the VSL gains drops to 1.131:1. And if people would demonstrate that they value other things by more than the VSL gains, it's hard for me to see the case for Policy B over Policy A.

Note: I've read only Grimes's summary and not the underlying work.