Thursday, 13 October 2022

Refereeing in public health journals

Posted by Eric Crampton

Refereeing in public health journals always seemed a bit iffy.

A few years back I'd noted a case where the NZ Med Journal published a piece with 9 co-authors on drinking in pregnancy, none of whom noticed that 3 authors at Superu had done a better job with the same data only a few years before. None of the authors noticed. The editor of the NZ Med Journal didn't notice. The referees are supposed to be subject experts who watch their area; none of them noticed. 

The paper's authors eventually put up a note acknowledging the existence of the prior Superu work. And, ever classy, when their note (published in NZMJ as a separate piece) refers to their own prior work, they cite it; when they mention the Superu paper, it's as an inline URL that citation metrics scrapers might not catch. 

And of course they went for the "our attention has been drawn" framing rather than pointing to any of my posts on it, which might risk pointing out that some economist knew the lit in their area better than anyone who was involved in the writing or publication of their original article.  

Rehashing that now because I noticed a tweet by Clive Bates, who's expert in vaping. 

Zvi Herzig, Clive Bates, and Peter Hajek provided a post-publication review of some work on vaping harms by the Otago public health folks. They found a pile of issues that the authors, referees, and editors over at BMC Public Health missed. 

For example, you shouldn't compare the rates of some chemical's presence among smokers and users of other nicotine delivery systems without putting in a control group who don't use either.

First, the authors ignore background exposures (the ambient exposure experienced by non-users arising from the environment, food etc.), with the exception of an attempt to correct for acrolein exposures (discussed below). This is a serious error because most of the biomarkers measured are present in nonsmokers at significant levels.[7][8][9][10] Consider this illustration: if smokers have a level of a given chemical of 60 and ENDS users of 20, the authors would assert that exclusive ENDS use poses a third of risks of smoking. However, if non-smokers also have the level of 20 for the given biomarker, ENDS use poses no incremental risk at all. 

In fact, in the Jay[6] and Hatsukami[4] studies (comprising 11 of the 17 comparisons), abstinence and NRT arms show near-identical outcomes to those of their ENDS arms, yet this important finding is not reflected in the analysis.

They also note that a lot of people classed as exclusive ENDS users were in fact smokers, among other problems. There isn't carbon monoxide in vape, so if you're seeing high CO levels, you probably have somebody who's still smoking.  

Wilson, Summers, Ouakrim, Hoek, Edwards and Blakely wound up publishing a substantial correction, retracting their prior conclusion that vaping is about a third as harmful as smoking. 

Which is great and all. But while they grabbed the references that Herzig et al pointed them to in issuing their correction, they don't anywhere acknowledge Herzig et al's assistance. 

Bates, on twitter, points to the original referee reports over at BMC Public Health.

at

No comments:

Post a Comment