Satisfaction (Treasury stakeholders can't get no)

So Treasury sat on its 2017 external stakeholder survey for a year. In February, it told itself that it would release the results of the survey when it had achieved sufficient progress in improving things. It provided me the survey this week in response to an OIA request, but who knows when they would have otherwise gotten it up on their website.* 

The 2015 version of the survey was up within a couple of months.

The survey results aren't all bad. But on the key area I've been worried about around Treasury's economic capabilities, they aren't good. We can apply our usual bit of skepticism where a lot of the respondents to these surveys would be other Wellington types who might have their own unreasonable reasons for being mad at Treasury, but that should be in the fixed effect.

I go through it in this week's Insights newsletter.

Among those people interacting with Treasury about its core business of economics, macroeconomics, and fiscal projection, satisfaction dropped from 70% in 2015 to 47% in 2017. The proportion of stakeholders viewing Treasury staff as well-informed dropped significantly, as did overall confidence that staff do a good job, that Treasury challenges thinking on critical issues, and that Treasury can offer insights.

One highlighted survey response noted that “Treasury staff are personally a pleasure to work with, but they don’t have a strong background in economic analysis.” It is no particular surprise that lifting the quality of staff was near the top of the list of things stakeholders wanted Treasury to focus on.

The August survey results were reported to the Executive Leadership Team in November 2017 and discussed in February 2018. This week’s letter from Treasury says they are working on building economic capabilities through measures like training existing staff and through recruitment.

When Treasury would have released the Stakeholder survey, barring being prodded, is anyone’s guess. Only four of fifteen hired by Treasury in the 2019 graduate recruitment round had at least Honours-level training in economics and finance; I doubt that counts as addressing the concerns raised in the survey.

The consequences of Treasury’s failed experiment in de-emphasising economics will be felt for a long time. Treasury needs again to be the place where top economics graduates want to work.

The OIA materials are all available at the link above as well.

I have another couple of requests now in; the OIA due date will be 4 October.

Treasury said that they're working on building economic capabilities through training and recruitment.

So I have asked for details on their in-house training programme, including any syllabus and curriculum, and any assessment or review that Treasury might have undertaken benchmarking knowledge provided through that programme against university training in economics.

On building capabilities through recruitment, I have asked for detail on the qualifications and fields of analysts, senior analysts, principal advisors and senior managers hired for each of the past several years, as well as the same detail on people leaving Treasury for the past several years. I know of a couple of great senior people who have joined Treasury recently, but I also know of other great people who have left. It would be nice to have a sense of that flow.

Finally, I've asked for more detail on Treasury's award-winning blinded recruitment programme.

I understand that the process also blinded the applicant's field of study, or at least that it did so for some recent years. I want to know more about that. 

I'd like to know:

• at what levels of recruitment it applied;
• over which years' recruitment it ran;
• what evaluation framework was set up when they implemented the thing and any evaluation that was subsequently undertaken;
• whether it really did scrub out the applicant's field of study
• and, if so, over what period;
• and, if so, what analysis formed the basis for deciding to do that, and what analysis was undertaken in assessing the effects of doing that;
• on what basis Treasury was able to choose among applicants if they knew nothing about the applicant's grades, degree level, or university; 
• what the applicant pool looked like in each of the last few years, what the pool of applicants extended interview invitations looked like, and what the pool of extended offers looked like (degree and majoring field). Basically, I want to know if economists stopped applying to work at Treasury or whether the blinding knocked econ applicants out, and what the time path there looks like. 

Stay tuned. Maybe they'll tell me.

* Update: the thing's on their website, but only listed as 2017. They don't say when it was undertaken in 2017. Colmar Brunton ran the survey in early August; responses were received by respondents from about 31 July, with a four-week response window. So all responses would have been in by late August. This was in the email I'd received from Treasury as a respondent, so you can see the timeline here:

What happens to the results?Colmar Brunton will analyse the results and provide a report to the Treasury in September.  The report will be published later on the Treasury’s website. Previous research has provided valuable information on what the Treasury needs to do to better engage our stakeholders.

Maybe they've only sat on it for 11 months if they received it late September. It's well out from 2015's 2-month turnaround.