Sunday, 7 March 2010

But they're all subject to manipulation...*.

Justin Wolfers lauds the new Oscar voting system, also described here. He claims:

Let's take Wolfers's example: 1997's win by Titanic over As Good As It Gets, The Full Monty, Good Will Hunting and LA Confidential. Wolfers says that someone expecting Titanic and As Good As It Gets to be the top picks would have an incentive to lie about his true preference for LA Confidential and to vote for his favourite among the two likely front runners. But mightn't folks have incentive to lie also under the elimination ballot?

Suppose we have 5800 Members of the Academy voting, with preferences as listed below.

The first set of voters, 2000-strong, prefers Titanic to anything else, followed by LA Confidential, followed by Good Will Hunting, followed by Full Monty, followed by As Good As It Gets.

The second set of voters, all 1400 of them, prefers Good Will Hunting followed by LA Confidential followed by The Full Monty, followed by Titanic, followed by As Good As It Gets.

The third set of 1000 voters prefers As Good As It Gets, followed by LA Confidential, followed by Good Will Hunting, Titanic, and The Full Monty.

The fourth set of 600 voters prefers LA Confidential, then Good Will Hunting, then Good As It Gets, then Full Monty, then Titanic.

The final set of 800 voters prefers As Good As It Gets followed by LA Confidential, Full Monty, Titanic, and Good Will Hunting.

On a straight plurality vote, Titanic wins with 2000 votes to As Good As It Gets's 1800.

Under a sincere elimination ballot, Full Monty is eliminated on the first round (nobody's first pick) followed by LA Confidential on the second (first pick of only 600), As Good As It Gets on the third (now the first choice of 1800), and Titanic on the fourth (now first choice of 2800), leaving Good Will Hunting as winner.

Is it then true that Good Will Hunting is preferred by a majority to any other film? Obviously not. LA Confidential is preferred over Good Will Hunting by all but the second group of voters: 4400 preferred LA Confidential, 1400 preferred Good Will Hunting. Indeed, LA Confidential is the Condorcet Winner in the preference ordering I've set up above. It's true that the winner on the elimination ballot is preferred by a majority to the film it beats out in the last round, but that's hardly the same thing as being majority preferred over all other nominees. I'm guessing that Wolfers here must have been misquoted.

As for manipulation, imagine that you're in the final group of voters and you've a decent idea of how things are going. You and the rest of your group will not be happy if Good Will Hunting wins: that's your last choice. How can you change the outcome? Lie about your preference between As Good As It Gets and LA Confidential! Doing so flips the outcome to LA Confidential, your group's second choice, from your group's last choice. The elimination sequence then runs Full Monty on the first round, Good As It Gets on the second (1000 votes), Good Will Hunting on the third (1400 votes), and LA Confidential beats Titanic in the final round, 3800 to 2000. Lying in this case helps in choosing a Condorcet Winner, so that's a decent result, but it doesn't have to be the case.

Now it's true that the informational requirements for productive preference misrepresentation under the elimination ballot may generally be higher than under plurality, but we can easily imagine rules of thumb that wouldn't be too bad. If your top-ranked option is likely to lose, push it down your preference ordering; if your last-ranked option is likely to win, claim to love anything that other folks are likely also to support. Since everybody else is likely to be doing the same thing, things will be messy. But that's also true under plurality. While Titanic won by plurality in our example above, it also was very low in everyone else's preference orderings. Titanic fans could easily imagine the fourth group supporting As Good As It Gets to stop Titanic, in which case they could flip to LA Confidential and encourage Group 2 to do the same; in that case, it's folks preferring one of the "big two" who do better by lying.

It's because social choice gets so messy that I went to the data to tell me which system gave better results. I can't see much difference between them in terms of retrospective critical evaluation of film quality. Don't get your hopes up too high....

*Except for dictatorship of course, unless reasonable homunculi disagree...


  1. If I'm not mistaken, however, Wolfers' statement holds true under the new system (but not the old system, under which you can win if 21% of people love your film and everybody else hates it) as long as everybody votes honestly. Given that manipulative thinking in the context of the Oscars is alien to the good people in Hollywood, everything is hunky dory.

  2. The new system is an elimination ballot, as best I understand it. Rank order your preferences. The option with the fewest first choices is dropped, with the second choice on those ballots becoming the new first choice. Note that the first example I gave had no manipulation but resulted in the Condorcet winner not being chosen...

  3. It's a common misconception. See the "What is this bill?" tab on the Green HSR Bill page (
    "a New Zealand head of state directly elected by the people via the STV preferential system (to ensure the successful candidate has a mandate from over 50% of the electorate)." I'm not sure STV or IRV is a very good system, and am doing a bit of research into it at the moment.

    PS, KiwiPollGuy here, still can't comment using my WordPress account, so using an old Google account instead. Don't have this problem on other Blogspot blogs, is there something wrong with Offsetting's settings?

  4. @KPP: I can't believe that Justin Wolfers would seriously have gotten that wrong; he must have been misquoted. Other folks getting it wrong, sure.

    On comments: Sorry! I'm using blogspot mostly because I'm an idiot about keeping the settings right, and WordPress seems very options-intensive. The only options I get on comments on Blogspot are

    1. Anyone (no...want to have a minimum hurdle)
    2. Registered users including OpenID (my pick)
    3. Users with Google Accounts (too restrictive)
    4. Members of this blog (far too restrictive).

    There aren't any other relevant options on comments, so I'm not sure what the problem is. Do you have an OpenID associated with your WordPress account?

  5. Eric,
    I have OpenID, and don't have any problems commenting on certain other Blogspot blogs (BKD's, for example.) Not sure what the problem could be.

  6. Bizarre. I've no clue why mine is giving you problems, but Drinkwater's blog takes anonymous comments.

  7. And it seems to allow my OpenID (see above)