Industry Insights

The editor-in chief at Public Health Nutrition makes a backhanded case for including industry as referees on academic papers.

As recap: that journal accepted a paper last June titled "Ultra-processed foods have the worst nutrient profile, yet they are the most available packaged products in a sample of New Zealand supermarkets".

Katherine Rich, of the Food & Grocery Council, wrote a letter to the editors of the journal noting substantial methodological problems with the paper. Most importantly, the authors did not seem to understand how the Nutrient Profiling Scoring Criteria works. Breakfast cereals, for example, are poorly treated in the NPSC data: a cereal that is, say, 50% fibre would show up as being 4.7% fibre. Much of the content is then missing in that data. Rich demonstrates how this would bias their figures.

The published letter is far more in-depth than most referee reports I've seen. Rich goes beyond identifying a potential problem to show exactly how that problem biases the results presented. It is not common to have a journal referee replicate part of your work, fix it, and show you the consequences of what you've messed up.

The journal then contacted the Chief Public Health Nutrition Officer at Food Standards Australia New Zealand to check whether Rich were right. And the editorial on it all says:

The subsequent submission of a letter from Dr Mackerras, the Chief Public Health Nutrition Officer at Food Standards Australia New Zealand (FSANZ), reiterated some of the same methodological concerns(15) and confirmed the value of addressing those concerns in detail.

The editorial notes that had Rich submitted her findings as a peer-reviewed article rather than as a letter, it likely would have "received closer scrutiny or been met with greater scepticism" than one received by an author without an obvious conflict of interest.

The editor goes on to try to justify one-sided scepticism, noting cases where industry has tried to skew things, and impugning Rich's character. But it seems a bit of an odd turn where Rich pointed out a problem that only an industry expert would have caught. Rather than thanking her for correcting errors, they highlight that Rich plays an advocacy role and criticise her for having published her critique on the FGC's website. Nowhere are the academics who wrote the paper criticised for not having checked into their data source.

As a side note, we were concerned that the methodological issues raised by Ms Rich and Ms Mackerras from FSANZ should have been caught during the review process and perhaps were missed because the findings confirmed what the reviewers hoped or expected. In this case, however, the comments raised by the three reviewers appeared to be quite balanced overall. The points raised by Ms Rich and Ms Mackerras were perhaps only obvious to people who were very familiar with and invested in the scoring system used in the paper. Our concern about ‘confirmation bias’ in the review process relates to the possible role of undisclosed, ideological interests – a second pitfall of disclosure statements.

I don't know how many times I've heard public health activists say that industry deserves no place at the table. But leaving them out systematically biases things. You mess things up when you deliberately leave out people who know things that you don't.

Imagine if journal editors considered including as referees industry folks who actually know what's going on in their area. The Editor always can ditch silly suggestions that come of it, and the industry referees would know that. But if an author's doing violence to the data in ways that only someone working with that data would know about, doesn't it make sense to, well, ask one of the industry people who works with that data?

I wonder how many in industry would take the time if asked.