Saturday, 30 January 2010

Tierney on conflicts

Zero skepticism about research results based on funding links: bad.

Total skepticism about all research results based on funding links: worse.

One-sided skepticism about any research you don't like because of funding links: worst of all.

Writes Tierney:
Conflict-of-interest accusations have historically been used selectively to marginalize dissenting opinions, as Gary Taubes nicely illustrates in his book, “Good Calories, Bad Calories.” Scientists who disagreed with the accepted wisdom on the evils of fat in the diet were accused of being corrupted by industry grants even if they had received most of their money from government agencies that were looking — unsuccessfully — for evidence to back the fat-is-bad theory. Meanwhile, scientists who went along with the conventional wisdom on fat weren’t criticized for the corporate money they’d received from food companies.

Mr. Taubes has also found some wonderful examples of selective journalism in the dispute over sugar’s health effect: An article stressing the harms of sugar would make dissenting scientists look bad by stressing their connections to the sugar industry, whereas an article exonerating sugar would make the other side’s scientists look bad by stressing the money they received from companies making sugar substitutes.

Meanwhile, Mr. Taubes writes, journalists have paid too little attention to the scientific questions or to other types of bias: “Scientists were believed to be free of conflicts if their only source of funding was a federal agency, but all nutritionists knew that if their research failed to support the government position on a particular subject, the funding would go instead to someone whose research did.” David Kritchevsky, a a member of the federal advisory board that issued dietary guidelines in the 1980s, summed up the pressure on researchers: “The U.S. government is as big a pusher as industry. If you say what the government says, then it’s okay. If you say something that isn’t what the government says, or that may be parallel to what industry says, that makes you suspect.”

Anther journalistic blind spot, Mr. Taubes argues, concerns not-for-profit advocacy groups like the Center for Science in the Public Interest, which campaigned for more regulation of the food industry and denounced scientists who accepted money from it. Such advocacy groups, Mr. Taubes writes, “are rarely if ever accused of conflicts of interest, even though their entire reason for existence is to argue one side of a controversy as though it were indisputable. Should that viewpoint turn out to be incorrect, it would negate any justification for the existence of the advocacy group and, with it, the paychecks of its employees.”
When Matt and I were writing on alcohol, we'd often be accused (woefully incorrectly) of having been funded by the alcohol lobby; by contrast, nobody much worried about the money paid to the consultancy firm that seemed to have been directed to come up with a very large number on the costs of alcohol.

The sad state of affairs is that anybody suggesting that alcohol or tobacco is anything less than completely evil will have all of his work entirely discounted if he's funded by industry, government won't fund anything that isn't likely to show alcohol and tobacco as being completely evil, and folks funded by government agencies or NGOs that demand findings of "EVIL EVIL EVIL!" are never thought to have anything but the most benign and non-pecuniary of motivations. Guess what that does to the level of debate over time when researchers also have to pay their bills?

Aim for two-sided moderate skepticism. If you're worried about the corrupting influence of big oil on some climate scientists, worry also about the corrupting influence of competing for research grants from pro-warming governments. Put more faith in results where the authors set things up for easy replication and less where they bury the data.

8 comments:

  1. Thank you, it's refreshing to hear someone voice this.

    One thing has always bothered me however: while the incentive for industry to distort scientific results is well-understood, it's harder to pin down incentives for governments or other elected bodies -- after all, what do they stand to gain? I suppose it's conceivable that they might push for results to be in alignment with public sentiment in order to increase their popularity and thus chances of reelection, but I find that answer a bit unsatisfying. If you have a better answer, or could point me to a good discussion it would be great!

    Thanks,
    Tim

    ReplyDelete
  2. Pretty simple model on this one, really. Stop thinking of government as a unitary actor. Health departments bear fiscal costs of alcohol use but gain few of the benefits, so they'd like to have tougher regulations on alcohol use because those regs are an off-budget expenditure that don't cost them much other than lobbying costs. So they commission studies that inflate the perceived costs of alcohol to push policy in their preferred direction via voter pressure.

    ReplyDelete
  3. Tim and Eric,

    I have a different model (based on bitter experience working at Health Canada). There is a lot of self selection amongst public servants in terms of which Department they are working at. Public servants at, say, a government health department very likley put more weight on good health compared to other pleasures of life compared to the general population, and would like to see policies that refelct that view. Knowing that the general population don't share the same mindset, they have an incentive to commission studies that can justify the policies they want.

    ReplyDelete
  4. No reason both can't be going on at the same time...

    ReplyDelete
  5. Thanks Eric and Seamus. Eric, you're certainly right to point out that government consists of different parts with (often) different agendas. But is there really pressure on a *public* health department to reduce expenditure? There's no possibility of competition. I'd even go so far as to suggest that a government that promotes itself as left-leaning might have a perverse incentive to spend more on health, to bolster its image as "the provider". (I know, more single-actor-think...) So I'm not (yet) convinced that parts of a government meet the requirements of being rational actors.

    Seamus, that sounds totally plausible to me. But I hope there's more to it than that, because that explanation invites the retort that self-selection surely operates in the private sector also, where it could be said to skew company directors toward the criminally greedy, ultimately resulting in inefficient allocation of resources. It's hard (I think) to believe that one of these self-selection effects could be pervasive, and the other negligible.

    Very interested to hear your thoughts!

    ReplyDelete
  6. @Tim: In a Niskanen model, a budget-constrained bureau has every incentive to keep costs down while a demand constrained one will inflate costs. Health is more budget constrained than demand constrained. I had Niskanen in the back of my head when writing.

    Seamus's response reminds me then of Niskanen's response about legislative controls: the high demanders get onto the committees....

    ReplyDelete
  7. Thanks Eric. As the most dabbling of dabblers in economics, we are now officially over my head :) Someday I should read up on Niskanen, as this topic ("Industry has incentives to be naughty, while government doesn't") is something that seems to come up frequently and I'd like to be better informed.

    Thanks!

    ReplyDelete
  8. The postcard version of Niskanen:

    It's a mistake to view government bureaus as all facing the same constraints. They all want to maximize discretionary budget, but some agencies are demand constrained - the public values negatively the marginal unit even if free - and some are budget constrained - total surplus from early parts of production are exceeded by excess costs from later parts of production before marginal gross value of the last unit turns negative.

    If budget constrained, a bureau will produce efficiently but will produce too much output (twice as much under linear systems); if demand constrained, a bureau will produce the right amount of output but at too high a cost. In both cases, the bureaus are optimally choosing their preferred budget/output combination; constraints vary.

    ReplyDelete