The alcohol levy review - an ongoing OIA saga

I keep a bit of a watching brief on the old BERL social cost of alcohol figure. It turns up in weird places. 

As aide memoire, BERL produced the number as commissioned work in the late 2000s that was meant to follow the method set by Collins & Lapsley in Australia. 

The Collins and Lapsley method has a few problems. But BERL compounded those problems with choices that seemed designed to generate a larger number for the tallied social costs. 

For example, Collins & Lapsley had aetiological tables that tried to attribute the fraction of different disorders that might be attributed to alcohol use. Their tables had a few disorders where the aetiological fraction was negative because drinking reduces the incidence of that disorder. BERL decided that, because they were only looking at harmful drinking, it was ok to just set all those cells in the table to zero rather than maintain a number showing benefits (and consequent reductions in net harm). 

Matt Burgess and I went through the BERL report, seeing what the number would look like if more standard method were followed. For example, BERL counted as social cost to the country every dollar spent on alcohol, including every dollar spent on excise, by those drinking more than about 2 pints of beer a day. Drinkers' spending on beer is a social cost only in the sense that private costs are part of social costs. And since benefits enjoyed by drinkers would need to be netted for any sensible net cost figure, the whole thing was a bit suspect. 

BERL responded to the critique by updating the figure to no longer count as a social cost drinkers' spending on alcohol excise, but let the rest stand. 

Brad Taylor joined Matt and me for an update to the review in 2011, when we went through the underlying Collins & Lapsley work. We adjusted upward the revised BERL figure, but the majority of the BERL-tallied costs were either double-counting or costs far better considered private than external and social. 

BERL provided an updated figure in 2018, but it turned out just to be the old figure multiplied by GDP growth over the period. Which could be fine if the initial number were sound (it wasn't) or if alcohol social costs scaled with GDP (they don't necessarily, and especially where alcohol consumption was declining over the relevant period). 

And the whole thing is a bit silly where the measured social cost really doesn't matter. The policy question is always whether any intervention, whether excise or otherwise, provides net benefits. Interventions can fail to do so despite very high measured social cost; they can also provide benefits even if social costs are low. The only reason for generating large social cost numbers is to motivate "something must be done" responses. 

Anyway. 

The number turned up again in last year's "Independent Review of the Alcohol Levy Stage 1: Rapid Review". The work for the Public Health Agency was undertaken by NZIER and Allen + Clarke. 

The work included this section:

90. The cost of alcohol-related harm to New Zealand society is significant. This section provides a summary of existing estimates of the cost of alcohol-related harm in Aotearoa New Zealand. 

91. The most recent study to quantify the social cost of alcohol in Aotearoa New Zealand was conducted by BERL in 2009. Commissioned by ACC and the Ministry of Heath, the report aimed to quantify the social cost of alcohol and drug related harm looking at the personal, economic, and social impacts. While the estimate of the social cost of alcohol-related harm in Aotearoa New Zealand published by BERL in 2009 and updated in 2018, or rather the methods used to generate it, have been criticised by some commentators, it has been widely cited in the alcohol-harm research and policy space in New Zealand over the last 14 years (BERL, 2009; Nana, 2018). The Law Commission’s 2010 report on the review of the regulatory framework for the sale and supply of liquor also cited the BERL 2009 report. 

92. In 2018, the updated estimate of the social cost of alcohol, based on the BERL methodology, was calculated to be $7.85 billion per year (Nana, 2018). This estimate included costs resulting from justice, health, ACC, social services, unemployment, and lost productivity. Intangible costs such as years of life lost from premature death, lost quality of life, child abuse, sexual abuse, and impacts on victims of alcohol-caused crime are also relevant to assessing the overall impact of alcohol-related harm on society. The 2018 update did not include intangible costs. A recent Australian Study found that in Australia $48.6 billion AUD of intangible costs could be attributable to alcohol (National Drug Research Institute, Curtin University, 2021). 

This section seemed particularly poorly undertaken. Citing the 2018 figure seemed particularly odd where the thing was just the old number multiplied by cumulative GDP growth. 

It's also incorrect to say that the 2018 update didn't include intangible costs. Intangible costs of lost life and lost quality of life were included in the 2009 figure, and the 2018 figure just inflated the old number by GDP growth.  

Paragraph 91 alludes to that 'some commentators' have criticised it, but said nothing about the nature of those critiques or who made them. Were the concerns trivial or notable?

Meanwhile, the bibliography included these two relevant references that weren't included in Para 91:

Crampton, E. (2018). The alcohol cost ‘zombie’ has returned. 

Crampton, E., & Burgess, M. (2009). The Price of Everything, The Value of Nothing: A (Truly) External Review Of BERL’s Study Of Harmful Alcohol and Drug Use (Working Paper No. 10/2009).

The 2009 piece was my original critique of the BERL figure with Matt; I'd have preferred the updated critique from 2011. The 2018 column had my initial guess that the updated BERL figure was just an inflation and population growth adjustment; my 2019 column had Ganesh Nana's confirmation that the new figure was the old figure inflated by cumulative GDP growth. So I'd have pointed to the 2019 column instead. 

But the authors clearly knew about my critiques. That they were in the bibliography suggested that there might have been more fulsome discussion of those critiques in earlier drafts. 

On 6 September 2023, I sent an OIA request to the Ministry of Health asking for all early and working drafts produced by NZIER [Paragraph 14 of the report said that NZIER undertook the analysis of existing data and evidence]; for correspondence between and notes from conversations between MoH, HPA, Allen + Clarke and NZIER regarding NZIER's analysis; and, for any peer review of the report. 

On 15 September, MoH replied saying that the correspondence would be extensive and that I needed to refine the request if I wanted to get anywhere. 

I replied immediately asking them to prioritise delivery of early and working drafts, and any peer review. I also suggested prioritising correspondence and relevant notes from meetings between and among MoH, HPA, and Allen + Clarke regarding the NZIER report. 

On 6 October, I reminded MoH that the refinement of my request only asked that they prioritise two parts of the request, and should not have triggered a clock reset; the requested information was due.

On 17 October, I had a reply from the Public Health Agency's Ross Bell. He noted that they'd considered the refinement as having triggered a time extension. But more substantively, they refused early and working drafts, as well as peer reviews, under 9(2)(g)(i) to protect free and frank expression of opinions. 

I proceeded immediately with the Ombudsman. 

On 16 November, the Ombudsman's Office commenced investigation. 

On 13 December, the Ombudsman's Office advised that the Ministry was prepared to reconsider its decision with respect to final drafts and asked whether that would be sufficient. I wouldn't know until I'd seen any released documents - if the released drafts let me see what had happened in the relevant section, that would be fine. If they didn't, I'd need to see more. I'd have to wait. 

On 2 February, a Senior Investigator at the Office of the Ombudsman noted that the Ministry had advised it would be providing a partial release, and asked whether I wished that they review the withholding of the earlier drafts; I noted that I couldn't know until I'd seen what they would release.

On 4 March, the Office reported that they were still chasing the Ministry about the later drafts. 

On 11 March, the Ombudsman advised that he had sent a letter to the Ministry recommending that the documents be released immediately and apologise for the delay.

At close of business on 14 March, the Ministry of Health released the later drafts. Ross Bell, Group Manager, Public Health Strategy & Engagement at the Public Health Agency, apologised for the delay and any related inconvenience.  

While those drafts did include some annotations from "KT" and Te Whatu Ora, they did not provide much light on what had happened with the section on alcohol social cost. The earliest draft was substantially similar to the final. 

So I still cannot really tell what happened. 

The bibliography references to the critiques suggest that, at minimum, those references were included as a citation in an earlier draft of Para 92. It's possible that an earlier version included more substantive discussion of those critiques, but it's hard to say.

I've asked the Ombudsman to form a determination around those earlier drafts' discussions of the costs of alcohol-related harm.

I suspected that the first draft from NZIER included substantive discussion of the relevant arguments. NZIER aren't idiots; they know this stuff. It's in the bibliography, so it was there at some point. 

If there had been more substantive discussion, was it excised at request of Allen + Clarke, or at request of the Public Health Agency?

In either case, the effect is a document sent to the Minister, advising on the alcohol health levy, that provides a fairly one-sided view on alcohol social costs. 

I yesterday received an additional bit from the Ministry, which might speak to the Public Health Agency's views on things:

Kia ora Eric,

Further to the below email sent to you containing the reconsidered documents of your OIA (ref. H2023031477), the Ministry has identified a paragraph pertaining to yourself in one of the early draft documents. While the Ministry is maintaining its position on withholding the early draft documents under 9(2)(g)(i) of the Act, the following excerpt is being released to you under section 16(1)(e) of the Act: 

So it seems that early drafts did include substantive discussion of my critique of the BERL figure, and that someone caused it to be erased.

I'd also note that I was discussant at the NZAE meetings on the BERL paper in 2009. It was standing room only, because my critique of the BERL paper had already been released. The Ministry could consider asking any economist in the room whether my critique was just a me-thing, or whether the profession broadly shared my concerns.

I did that work as an academic in the Department of Economics at Canterbury, five years before I joined the Initiative, and two years before doing any industry-funded work. The funded 2011 work [funded by NABIC] discovered an error in the earlier unfunded work that had us revise upward the earlier estimate of alcohol social cost. 

I note that Ross Bell, now relevant Group Manager at the Public Health Agency, was Executive Director of the Drug Foundation when the BERL figure was originally being critiqued. 

Here is the issue of the Drug Foundation's "Matters of Substance" newsletter that included discussion of the controversy around BERL's number. It would be surprising if Bell were not aware of the difficulties with BERL's figure. He had the masthead editorial on the issue of their newsletter in which my critique of the BERL figure was discussed. 

I'll look forward to seeing whether I can get any further with this via the Ombudsman. 

In the meantime, it looks pretty obvious that the Public Health Agency was very happy to put a biased document up to the Minister as advice - whether they requested that outcome directly, or had Allen + Clarke do it.

A provisional health warning on advice from the Public Health Agency may be in order. At least until we can figure out what the heck is going on over there. 

And a reminder that government-commissioned reports face censorship regimes. If the Ministry doesn't like what it says, well, the offending bit gets disappeared. As an offending bit here seems to have been disappeared.