Peer review and error rates

Inside Higher Ed points to a 30-year old study I'd not seen:

Although peer review is often portrayed as an institution that arose with the scientific method, Fitzpatrick suggests the roots of peer review were “more related to censorship than to quality control,” serving mainly to concentrate academic authority in the hands of journal editors and, later, their expert reviewers.
While this system created an effective supply-side filter, it was also susceptible to bias, as Douglas Peters and Stephen Ceci demonstrated in a 1982 experiment. Peters, of the University of North Dakota, and Ceci, of Cornell University, took already-published articles in 12 esteemed psychology journals and resubmitted them, changing only the authors’ names and affiliations and the phrasing of the opening paragraphs. Three of the 12 articles were caught by journal editors as duplicates. Of the nine that were not, one was published. The other eight were rejected, most on methodological grounds.

You'd think that's the kind of study that wouldn't now work - it would be too easy for a referee to catch plagiarism on a quick Googling. But maybe not. Nevertheless, I'd expect human ethics committees to object to the deceit involved on the part of the researchers, to deem the deceit and the cost imposed on external reviewers to be too perilous, and to kill the study; replication nowadays seems unlikely.

I wonder if the result is that surprising. Filters are set such that it's better to reject a whole lot of good articles rather than to let an unsound one through; if the signal is noisy, we'd expect that few articles would make it through twice. The experiment shows that articles by prestigious folks get rejected when re-submitted with a less prestigious affiliation, but without a control group of articles originally written by folks from less prestigious place, we can't tell whether the result is just what we'd expect from running things through the filter twice or if it demonstrates bias favouring those affiliated with prestigious institutions.

The rest of the Inside Higher Ed piece features interview with Kathleen Fitzpatrick on her recent book advocating changes in the academic publishing model; it's worth reading.