Wednesday 5 October 2022

Reader mailbag: the tricky maths of being inclusive

A loyal reader sends through a bit of math checking one of the standard bits of boilerplate on policy reports around climate change and adaptation strategies. 

I've seen the claim all over the place too. It's one of those things that feels like it was never meant to be taken literally, while simultaneously hinting that it should be taken literally. 

What does disproportionately even mean? If it isn't pinned down before someone goes in to try to make sense of it, it'll turn into a motte-and-bailey thing. 

My reader writes [with minor edits from me to anchor links]:

The tricky maths of being inclusive 

Adapt and thrive: Building a climate-resilient New Zealand – New Zealand's first national adaptation plan was published on 3 August 2022, and released on their website.

The webpage contains the extraordinary claim that:[1]

No two communities will experience climate change in the same way. Communities that are less able to adapt and disproportionately affected by climate change – including Māori, Pacific people and ethnic communities, low-income groups, disabled and older people, women, children and youth, and rural communities – are considered throughout this plan.

Why do I say this claim is extraordinary? Well, leaving aside the fact that the report does not contain any data to back it up, the claim is all but impossible mathematically.

Exploring further:

First, we need a scale on which the effect is measured. In the context of climate change adaptation, let’s call this scale expected adaption costs (A). This scale starts at zero (no expected costs), with positive values representing larger expected costs.

Second, to say that a group (G) is “disproportionately affected” the average A (Ā) for individuals in G has to be different from the average effect on individuals in the population (P) from which the group is drawn. I will use G to mean all people in the communities listed in the claim.

Third, the plan asserts that the communities G will be more adversely affected than others, i.e. ĀG > ĀP.

Fourth, let’s define another group R, consisting of everyone else, i.e. those in P who are not in G. Simple maths tells us that ĀR < ĀP < ĀG. That is, the average effect for R has to be less than the population average. This means that R is also “disproportionally affected”, just in the opposite direction from the effect on G.

If G had a small number of members relative to P, then in most situations ĀR would be roughly equal to ĀP.[2] In that case we could ignore the disproportional effect on R for practical purposes. But what about a situation in which most of the population is in G? In that case, the advantageous effect on those in R must be of a significantly greater magnitude than the adverse effect on those in G.

Let’s make a rough stab at working out the numbers in P, G and R. I’ll call them p, g and r.

Using the 2018 census data in NZ.Stat,[3] p = 4,699,755 (usually resident population count).[4]

The NZ.Stat standard breakdown by personal income, ethnic group, age and sex gives us a first estimate of r.[5] If I use total personal income below $30,000 as the “low-income groups” criteria, and age below 30 years as the “children and youth” criteria[6],  age 65 years and above  as the “older people” criteria, and everything other than “NZ European” and “New Zealander” as the “Māori, Pacific people and ethnic communities” criteria, then I get a first estimate of r = 582,210 (i.e. 12.39% of p).

But some of those people live in “rural communities”. The NZ.Stat subnational population estimates[7] for June 2018 allow us to estimate the proportion of males aged 30-64 that live in rural areas: 12.63%. This allows us to refine the estimate of r = 582,210 * (1 - 0.1263) = 508,677 (i.e. 10.89% of p).

But some of those people are disabled. The NZ.Stat disability tables for 2006[8] allow us to estimate the proportion of males aged 30-64 with “European” ethnicity that have a disability: 16.93%. So, my further-refined estimate of r = 508,677 * (1 – 0.1693) = 422,558 (i.e. 8.991% of p).

The claim doesn’t come with definitions, so I’ve had to make some assumptions. Further, the data on which I’ve made these estimates does not exactly line up with those assumptions. In particular, I’ve excluded people aged 25-29, which will bias the estimate downwards. And I can’t rule out some interactions between the variables I couldn’t control for when estimating the proportions in rural communities and with a disability.

Accordingly, I‘ll call the final estimate of 9% “about 10%”. So, the claim of disproportionate (adverse) effect applies to about 90% of the population. And, following the maths outlined above, the required disproportionate (advantageous) effect has to be around 10 times stronger.

Is this realistic? We’re talking about average effects here. Seeing as R is made up of middle-aged males with no recorded ethnicity, a middle-to-high income, living in an urban area, and without a disability, you’d think they were disproportionately more likely to own a home and to live near the coast. At least some of R are in the firing line for rather large climate-adaption costs. To maintain the group average for R, those coastal homeowners will need to be balanced out by others facing an even stronger advantageous effect. This will most likely require a subset of R to be facing negative climate-adaption costs – i.e. receiving a net positive benefit from climate change. This seems inimical to the Secretary for the Environment’s opening message in the report:

New Zealanders are already feeling the impacts of climate change… More change will come, and impacts will increase, disrupting nature and society, affecting people’s health and wellbeing and damaging livelihoods.

To conclude, the report makes a very strong claim of where climate-change adaptation costs will fall, which is mathematically implausible, perhaps impossible. The report contains no data to back up the claim, and the reasoning behind it is scant. Did the report go through a quality-assurance process? If so, why wasn’t this issue picked up? Are there other clangers in the report? Should readers trust anything it says?

[1] This claim, or a close analogue, is also made in the report’s text, but is less clearly stated.

[2] The exception would be if the effect on G was extreme.

[4] NZ.Stat: Age and sex by ethnic group (grouped total response), for census usually resident population counts, 2006, 2013, and 2018 Censuses (urban rural areas).

[5] NZ.Stat: Total personal income, work and labour force status, and ethnic group (grouped total responses) by age group and sex, for the census usually resident population count aged 15 years and over, 2013 and 2018 Censuses (RC, TA, DHB).

[6] The standard NZ definition for “youth” is < 25 years. See, e.g. https://www.myd.govt.nz/documents/policy-and-research/policy-document-final.pdf. Unfortunately, the age categories available in NZ.Stat don’t align nicely.

[7] NZ.Stat:

Subnational population estimates (urban rural), by age and sex, at 30 June 1996-2021 (2021 boundaries)

[8] NZ.Stat: Disability status by place of residence, age group, sex and ethnic group, 2006 

No comments:

Post a Comment