Monday, 14 October 2013

How'd they reckon this?

Americans surveyed in 2011 substantially overestimated the proportion of Americans identifying as homosexual. Where most estimates reckon about 3.5% of the population are homosexual, Americans surveyed thought that somewhere between 20-25% of the population are gay or lesbian.

Some candidate hypotheses for the overestimation:
  1. Availability bias where observations of people you know carry less weight than observations from TV shows or movies: if people take pop culture as more representative of average reality than their own personal circumstances, and if homosexual characters are over-represented on TV, then this could do it. 
    1. In that case, we would expect overestimation particularly among lower-IQ cohorts. 
    2. This alone shouldn't account for it: how many popular TV series other than Modern Family have at least 20% gay characters?
  2. Availability estimation of proportions where individuals of different characteristics are more or less likely to have friends or acquaintances who are gay. This would predict dispersion of estimates but shouldn't affect estimates of the population mean unless it's combined with downward bias in the number of people you know. If you're asked "What proportion of the population is gay or lesbian", and you think about how many homosexuals you know, and you then underestimate the number of heterosexuals you know, you'd bias upwards your estimate. I still can't see how that gets you to a 6-times overestimate.
  3. Ideology doesn't give clear-cut predictions, or at least not to me. You could build a story where social conservatives' fear of the 'gay agenda' is driven by their overestimation of that group's proportion in the population, or you could build an equally plausible story where social conservatives' dismissal of gay rights is founded on that the needs of a tiny proportion of the population should not drive changes in the definition of institutions that have persisted for thousands of years.
I'm putting most weight on #1. Gallup provided some population cross-tabs that can help:
eg5gc2qotkgwibicjrsenw.gift1eraykock-8vaxtp2ls6a.gif

Everyone overestimates, substantially. It's so far out of whack with reality that you wonder whether it's just a wonky survey. But the numbers are apparently consistent with the overestimates in a similar 2001 survey. 

Smarter and richer people, and men, have far more accurate estimates - this isn't out of line with fairly standard findings on other kinds of knowledge. Older cohorts were more accurate. 

Republicans, conservatives, and social conservatives were more accurate, which is inconsistent with the ideological hypothesis that "gay terror" would lead to overestimating the proportion of homosexuals in the population. And while "we shouldn't change everything for a small minority" would be consistent with social conservatives having a lower estimate than social liberals, which is true, it is not consistent with social conservatives still overestimating the population proportion more than five times over. 

Intriguingly, while social liberals more greatly overestimate population proportions, those favouring bans on gay and lesbian relations overestimate population proportions relative to those believing that gay and lesbian relations should be legal. This is likely (hopefully) an artefact of very small proportions of the population believing that gay and lesbian relations (not marriage, but relations) should not be legal. 

The data seems to give weak support to my candidate hypothesis #1, though it is completely indistinguishable from a dozen potential alternative hypotheses about intelligence, education, and accuracy in estimating things. It would be interesting to partial out the effects of education, age, gender, income, partisanship and ideology; alas, they give cross-tabs instead of regression coefficients.

Suppose that you favour gay rights, as I do. Would accurate perceptions of population proportions tend to increase or decrease support for gay rights? The estimate among those favouring same-sex marriage is just a titch higher than that among those opposing it, but at the same time college grads and postgrads have a smaller degree of overestimation and, I would expect, are more likely to support same-sex marriage. Only the partial derivative of the overestimate on the likelihood of supporting same-sex marriage in a probit would tell for sure.

Update: Chris Auld very helpfully points to work suggesting that correcting for under-reporting could roughly double the number. The sample in the paper is not representative, so we shouldn't extrapolate from their reported levels, but the magnitude of under-reporting is plausible. But even if under-reporting got us all the way to 10% in the full sample (7% seems more likely), that's still miles away from 20%.