Monday 8 June 2009

On the sensitivity of rankings when N is small

It's a bit of a running joke how sensitive economics department rankings can be to the location of the author of the ranking. David Anderson and John Tressler have found a different kind of sensitivity in rankings of New Zealand economics departments: how you treat the contributions of regular departmental visitors. This matters in small countries. Peter Phillips is a regular visitor at Auckland. Depending on the weighting scheme used for ranking the various journals, Peter Phillips personally accounts for between 9.6% and 56.8% of the total output of quality-adjusted pages produced in New Zealand economics departments (139 economists in total). If Phillips is not included in per capita output rankings, Auckland counts as 4th overall. If he counts at 10%, Auckland rises to 3rd. At 40%, Auckland rises to 2nd. If all of Phillips' contributions count for Auckland, Auckland moves up to first.

Long story short: it's not great to be graded on a curve that includes Phillips. I'll avoid other Phillips curve puns as William Phillips != Peter Phillips.

Canterbury comes in second in the per-capita rankings, with Otago in the lead. But we've made some very nice hires since the survey.

No comments:

Post a Comment