On Stillman SHOREs

So, did reducing NZ's alcohol purchase age really hike youth accident rates? SHORE says so:

Now, in the first long-term study, Massey University researchers Dr Taisia Huckle and Karl Parker have found this increased risk has become the new normal.

In the years leading up to the change, drivers aged 18 or 19 had roughly the same chances as those aged 20 to 24 of having an "alcohol-involved" vehicle crash that caused injury or death.

That increased in the years following the change, putting the younger drivers at 15 per cent greater risk in the first six years, then at 21 per cent greater risk up to 2010.

Loyal readers will recall Steve Stillman's work with Stefan Boes showing the opposite: they concluded that there was no increase in crash risk for 18 and 19 year olds after the purchase age change.

So why the difference?

First, the two studies are looking at different things. Huckle et al at SHORE use the ratio of alcohol-involved to non-alcohol crashes as dependent variable. They argue that this helps to control for the rate of non-alcohol-related crashes. They look at how this ratio for 18-19 year olds changes as compared to the similar ratio for older cohorts after the crash. If the ratio increases, that's potentially a form of difference-in-difference analysis that says something about the effects of the law change. They found that the ratio increased for youths relative to older cohorts.

Stillman and Boes instead looked at the number of alcohol-related crashes among 18-19 year olds as compared to alcohol-related crashes among older cohorts. They found no increase in the number of alcohol-related crashes for youths relative to adults once you put in appropriate time trends.

One problem with using ratios as dependent variable, and especially when you're using it in difference-in-difference comparison to another ratio, is that you can't easily tell whether any significant effect of the intervention was due to changes in alcohol-related youth accidents, non-alcohol-related youth accidents, alcohol-related adult accidents, or non-alcohol-related adult accidents. If non-alcohol-related accidents changed for other reasons around the time of the policy change, and if youths responded to that differently than adults, then that could have driven results.

Because the ratio measure isn't clean, it's then harder to talk about the "risk" of an alcohol-involved crash for youths subsequent to the change. You could say that the ratio has changed, but it's hard to say why. Stillman's paper pretty clearly shows that the "why" wasn't an increase in alcohol-involved accidents among 18-19 year olds.

Further, and as Thomas Lumley points out, while the paper says there was no effect on fatalities, each tragic anecdote in the Herald piece involved a fatality; the Herald story nowhere mentioned that SHORE found nothing on fatalities. The SHORE paper said "Lowering the purchase age had no impact on fatal alcohol-involved crashes among drivers aged 18 to 19 years compared with drivers aged 20 to 24 years." I'm a bit curious why the Herald then chose to lead with "Lowering the alcohol purchase age has been linked to a long-term increase in the chance of drunk drivers aged 18 or 19 being involved in car crashes that cause death or injury."