Tuesday 15 July 2014

Stat Juking revisited

I'd reckoned you'd need a bit of stats-fu to find evidence of police juking of the crime statistics. Turns out there was an easier way. Bevan Hurley reports that the Herald on Sunday got a copy of a report showing that Counties Manukau police had been fiddling the burglary numbers by recoding burglaries as less serious offences. 
About 700 burglaries were “recoded” in the Counties Manukau south area over three years, an internal police investigation has found. It found that about 70 per cent of the time, the offences should have remained burglaries.
The revelations will be an embarrassment for Police Commissioner Mike Bush, who was district commander of the area at the time, although he was not responsible for overseeing the coding.
Police have not said why the statistics were altered, but say staff were not under instruction to do so. Tolley denied police were under political pressure to reduce burglary statistics.
You don't need overt political pressure to get this kind of outcome, just KPIs with strong enough incentives. On the plus side, they were caught. On the down side, I can't see how lower level staff doing the coding would have any incentive to muck the stats around unless they were getting pushed by those whose KPIs did provide such incentive. It would be really interesting to read the full report.
The review listed dozens of examples where break-ins and attempted burglaries were downgraded, including one case where police failed to follow up after a witness gave them a burglar’s registration number.
The review found the burglary recoding rates in Counties Manukau south at the time were 15 per cent to 30 per cent whereas other areas typically recoded about 5 per cent.
So where last week's rumours were about failing to pursue charges, which wouldn't have mattered for stats based on recorded complaints, downgrading the complaints to less serious offences would matter.

I'd be curious to know what kinds of lesser offences were artificially inflated to keep the burglary numbers down.

I hope that the Police stats units have informed any researchers who'd been using the incorrect figures of the updated and corrected series. Anything that relied too heavily on 2009-2012 Manukau data is now going to have to be re-done.

The Herald on Sunday broke the story on the 13th. Their version is gated. The Stuff version, which notes "It was reported" rather than crediting the Herald, is here.

3 comments:

  1. Police statistics have always been a trap for the unwary. I recall doing some work years ago regarding the time at which crime occurred. All crimes have an 'occurred time' but in many cases it is an estimate - calculated as halfway between the discovery of a crime and when the crime-scene was previously seen. (e.g leave your car at the railway station at 8.00am, get back at 4.00pm and discover it has been broken into - Crime occurred at Midday). This might not be so bad if known times were separated from estimated time. This story is interesting and as is often the case - something malign is implied (by words such as 'manipulated') whereas the usual causes such as poor training, poor quality control, and incompetence are seldom inquired about - by the press at least.

    ReplyDelete
  2. I wouldn't be surprised if this wasn't targeting stats so much as workloads on an individual level. Burglaries are substantially more costly to follow up on in terms of Police time and resources. Some of this is the nature of the crime itself, but a lot of it is Police procedures.


    For instance, physical attendance at burglaries is mandatory, and it involves substantially more paperwork, follow up with victims etc, than does say a theft (which is often dealt with over the phone). At a high-level, this is for good crime science/victim-centric reasons. However the result can be significantly more work, for little to no increase in the the likelihood of actually solving the crime.


    You can imagine an individual officer/section being presented with a marginal burglary (say a break-in to a shed), where procedure dictates much work, for little gain, deciding to recode.


    Over three years, in a district like Counties Manukau, this could certainly amount to somewhere in the neighbourhood of 490 cases.

    ReplyDelete
  3. That doesn't seem at all implausible.


    We just need tests then for juking versus opportunity costs. In the juking case, we'd see more improper recoding of crimes for those ones that were nearing KPI thresholds, if there were explicit ones, or if rates were getting out of line with other districts, if there were only implicit KPIs. In the opportunity costs case, we'd see more improper recoding of crimes where time pressures from other crimes are binding. So juking would have local district burglary rates responsive to other districts' burglary rates; opportunity costs would have them responsive to other higher marginal value of effort crimes.

    ReplyDelete