No it didn't!

The Financial Services Council's Peter Neilson says Treasury's analysis of Kiwisaver's all wrong:

Neilson said the NZIER report found the evidence for Treasury's argument was too narrow because it used data only from the global financial crisis years.

He said it did not consider that KiwiSaver attracted young and low-income people who would not usually have been involved in formal savings schemes.

"The analysis simply compared the results for the people in KiwiSaver with those who were not, as opposed to those in the target audience who joined KiwiSaver compared with those in the target audience who did not."

"We need to compare apples with apples. People on a benefit can't afford to save and are likely to receive a higher income from New Zealand superannuation than they received during their adult lives on a benefit anyway.

"At the other end of the scale, people who were saving for retirement by investing in rental property or a farm would be unlikely to use KiwiSaver other than to just pick up the KiwiSaver incentives.

"For this group KiwiSaver would probably not increase their savings, it would only change the composition of their savings. Neither of these categories were in the target group for KiwiSaver and should not have been used for comparison," Neilson said.

I haven't read the NZIER piece yet. But I'm familiar with Treasury's work in the area.

1. While the Treasury's 2011 work was based on a 2010 sample, their more recent 2014 work was based on SoFIE and IRD data covering 2002-2010. They have a panel of 10,000 individuals from 2002 through 2010. 
2. The 2014 paper uses a difference-in-difference analysis looking at those who joined Kiwisaver as compared to those who didn't; they also ran diff-in-diff after sorting by age, gender and the like.  
3. The point of difference-in-difference is to let you compare those who joined with those who didn't in a way that's meaningful. The differences in the savings rates for the two groups in the period before Kiwisaver forms your baseline; the differences in the savings rate afterwards forms your treatment effect. Sure, there can be plenty of differences between the two groups. But those underlying differences are caught in the first differencing in the difference-in-difference. It somewhat odd to critique a difference-in-difference analysis for just comparing two groups. 
4. They found that KiwiSaver members accumulated less wealth than non-KiwiSaver members, correcting for other stuff. This is in the difference-in-difference: those joining KiwiSaver accumulated less wealth than those not joining, as compared to how both were doing before KiwiSaver.
5. If you want to restrict analysis to the ones that are really targeted by Kiwisaver and evaluate it on that basis, that's way different from a standard "was this programme a good idea" analysis. Think of it this way. Suppose that there's some terrible disease. One person in a million gets it. The only cure is getting a vaccine at birth. The vaccine costs $100,000. A cost-benefit assessment looking at the programme as a whole will say it's a colossal waste of money: you don't spend a hundred billion dollars ($100,000 * 1 million people treated) to prevent one instance of a terrible disease. But if you looked at it only on a target audience perspective - the one guy who'd have gotten the disease, then it's worthwhile: $100,000 to save that life was worthwhile. It's still a pretty bad programme on the whole though.

I will have to look up the NZIER report; it has to be better than what's here reported.

Update: the NZIER report is here. On first cut, it seems very odd to hang a lot on the behavioural economics literature around people screwing up savings when recommending a programme the default products of which are often entirely wrong for the person directed into them.