Blogs and peer review

I had coffee with Bryce Edwards over the weekend. His two year old is a week younger than ours; good times were had by all. But we both pondered what the New Zealand academic evaluation mechanism - the Performance Based Research Fund - will make of our blogging when we're up for evaluation in 2012. Are we entirely producing public goods (or, worse, consumption goods) at high opportunity cost? Are there so few New Zealand academic bloggers because we're the ones who haven't gotten our heads right?

PBRF requires that a peer review mechanism is in place certifying the quality of whatever you're doing. Blogs are a tough fit in that framework. They're continually reviewed post publication rather than one-off reviewed pre-publication. And instead of having anonymous, private referee reports that can be binned before sending the piece off to the next journal on the list, blogs have critiques that live publicly and forever. I've put this blog through what review processes I can: it's one of the fifty or so economics blogs recommended by the American Economics Association.

With that in mind, I read with interest Rajiv Sethi:

In fact, the refereeing process for blog posts is in some respects more rigorous than that for journal articles. Reports are numerous, non-anonymous, public, rapidly and efficiently produced, and collaboratively constructed. It is not obvious to me that this process of evaluation is any less legitimate than that for journal submissions, which rely on feedback from two or three anonymous referees who are themselves invested in the same techniques and research agenda as the author.

I suspect that within a decade, blogs will be a cornerstone of research in economics. Many original and creative contributions to the discipline will first be communicated to the profession (and the world at large) in the form of blog posts, since the medium allows for material of arbitrary length, depth and complexity. Ideas first expressed in this form will make their way (with suitable attribution) into reading lists, doctoral dissertations and more conventionally refereed academic publications. And blogs will come to play a central role in the process of recruitment, promotion and reward at major research universities. This genie is not going back into its bottle.