Hidden meanings

It's usually Alex that gives a plain English interpretation of something that Tyler only hinted at. This time, it's Alex that buries the point. He writes:

Are IVs Going the Way of the Atlantic Cod?

It's hard to come up with a good instrumental variable (plausible source of exogenous randomization) so when someone does come up with one (e.g. legal origin) it's tempting to want to use it again and again. Unfortunately, as Randall Morck and Bernard Yeung point out in Economics, History and Causation, IVs with more than one use are deeply problematic. If a variable is a good IV for X then it can't also be a good IV for Y without also controlling for X and vice-versa. What this means is that every new use of an IV casts doubt on every previous use.

As Alex well knows, if these latent variables are a problem, they're a problem regardless of whether the additional IV studies are conducted or published.

But he still points to a Tragedy of the Commons and suggests overfishing. What then is the Commons that is exploited when the kth paper is published using a well-used instrument? It isn't the technical validity of the instrument. And it isn't the soundness of prior work using that instrument. Rather it's the continued plausibility of the instrument at all - whether we still believe in it. Unintentionally helping reveal that certain prior results are less sound, I would have thought, is more a public good than a public bad. So what's then the Commons that's being overfished? Perhaps the plausibility of results reliant on complex econometric techniques as a whole. Overfishing it brings the whole enterprise into disrepute.

And so maybe Alex's hidden message is that a lot of empirical results are a sham. The Commons is public credibility. And the private incentive is to publish the incremental result that gets you another line on the vita but helps slowly to bring down the whole edifice. I'm not sure that anybody has successfully answered DeLong and Lang.

Karl Smith also suggests we ought to be raising the status of good narratives.

I tend to be pretty skeptical of results that aren't grounded in basic price theory or that aren't confirmed by a lot of different methods, including both Ocular and Ordinary Least Squares.