Wednesday 16 February 2022

Broken Data Pipes

This week's column in Newsroom went through the burst data pipe at Statistics New Zealand. NZ.Stat was shut down, abruptly, on 4 February. It's the only way of accessing some core statistics, other than the interim measure now in place of filling in a form and waiting for someone to send you the spreadsheet you were after.

It's the culmination of years of neglect of core systems. 

A few of us are looking at setting up an external mirror of Infoshare, in case SNZ decides they have to kill that even-more-archaic system too. 

After the front fell off of NZ.Stat, one informed data analyst, David Friggens, reported that the system was running on software that was five versions out of date. NZ.Stat is built on OECD.Stat. The Australian Bureau of Statistics, by his report, has version 9 of OECD.Stat in beta. And while Statistics NZ had recently started a project to upgrade to Version 10, the version of NZ.Stat that failed was Version 5.

I asked Statistics NZ whether Friggens was correct. Statistics NZ confirmed, via email, that they have been running “a legacy version of OECD.Stat software, with a project underway to upgrade this”, but did not specify which version had been in use.

To its credit, Statistics NZ had recognised some of the risk it faced. The Agency’s Statement of Strategic Intentions 2021-25 set workstreams aimed at ensuring “core information technology systems are at less risk of failing”. It sought to “identify risks to core systems and track the effectiveness of mitigating actions to ensure the stability of these systems.”

Unfortunately, it came a bit too late.

For years Statistics NZ has, like the proverbial local government, chased after shiny new objectives while largely ignoring the critical infrastructure that is necessary to keep the whole ship running.

The problem is not just budgets.

The problem is also priorities. 

No comments:

Post a Comment