Wednesday 3 March 2010

Social Choice and the Oscars

Colby Cosh reports that the Academy of Motion Picture Arts and Sciences is switching from a first past the post ballot for best picture to single transferable vote. I like this, mostly because I go over that system in my public choice class and sometimes get asked if anybody in the real world uses the system. Now I'll be able to say "Well, the Oscars". That's more fun than saying "Well, the Australians".

Voters will rank order the ten nominated films. If any film has a majority of voters picking it as top choice, it's elected. If not, the film with the fewest first place votes is eliminated and that film's supporters are counted as voting for their second choice. Repeat until some film has majority support. Writes Cosh:
Even in the old five-nominee Oscar races, a winner in the single round of voting could conceivably win even though 25 or 30 per cent of the voters loved it and everybody else hated it. (This may have enabled some Best Picture wins later considered controversial, like that of Crash in 2005 or Shakespeare in Love in 1998.) In a 10-nominee field, a mere 15 per cent might be more than enough. The board of governors, mindful that preferential voting was used in 10-nominee Best Picture races from 1936 to 1943, decided to re-adopt the old prophylactic measure against unpopular winners.
It's possible that the old system was more manipulable: if you could get about a fifth of voters to agree on a common first choice, you could potentially get it through despite a majority hating the film. That doesn't worry me too much: I liked both Crash and Shakespeare in Love. What worries me more in the old system is a plurality of uncoordinated voters with mediocre tastes coordinating around some unimaginative Schelling point (Avatar!). So I'll expect fewer oddball good picks like Crash but also fewer lukewarm picks.

I am rather curious about Colby noting the change in voting system at 1943. It would be really neat to compare current evaluations of Oscar winners for the period before and after the change as a measure of which system did better in picking the films later viewed as being very good. Let's give it a shot. Wikipedia lists all the winners and nominees; IMDB and Rotten Tomatoes score the films. A quick flip through doesn't show a clear winner across the two systems, mostly because both fare horribly. I apologize for the table formatting below.

YearWinnerBested by (IMDB)Bested by (Rotten Tomatoes)
1936The Great ZiegfeldDodsworth, Libeled Lady, Mr. Deeds Goes to Town, San Francisco, The Story of Louis Pasteur, A Tale of Two Cities (6 of 9 other nominees)Dodsworth, Libeled Lady, Mr. Deeds Goes to Town, Romeo and Juliet, San Francisco, A Tale of Two Cities (all rated nominees)
1937The Life of Emile ZolaThe Awful Truth, Captains Courageous, Dead End, The Good Earth, Lost Horizon, Stage Door, A Star is Born (7/9 other nominees)As IMDB, 7/7 rated nominees
1938You Can't Take It With YouThe Adventures of Robin Hood (1/9 other nominees)The Adventures of Robin Hood, Boys Town, Four Daughters, Grand Illusion, Jezebel (5/6 rated nominees)
1939Gone With The WindMr. Smith goes to Washington, The Wizard of Oz (2/9)Ninotchka, Of Mice and Men, Stagecoach, The Wizard of Oz, Wuthering Heights (5/9)
1940Rebeccanonenone, though many tie
1941How Green Was My ValleyCitizen Kane, The Little Foxes, The Maltese Falcon, Sergeant York (4/9)Citizen Kane, Here Comes Mr. Jordan, The Little Foxes, The Maltese Falcon, Suspicion (5/6 rated)
1942Mrs. MiniverKings Row, The Magnificent Ambersons, Random Harvest, The Talk of the Town, Yankee Doodle Dandy (5/9)As IMDB, and also 49th Parallel, The Pied Piper, The Pride of the Yankees, Wake Island (9/9: Every other film rates higher on Rotten Tomatoes)
1943CasablancanoneThe More the Merrier (1/6 rated)
1944Going My WayDouble Indemnity, Gaslight, Since You Went Away (3/4)Double Indemnity, Gaslight (2/3 rated)
1945The Lost Weekendnonenone
1946The Best Years Of Our LivesIt's a Wonderful Life (1/4)Henry V, The Yearling (2/4)
1947Gentleman's AgreementThe Bishop's Wife, Crossfire, Great Expectations, Miracle on 34th Street (4/4)Great Expectations, Miracle on 34th Street (2/4)
1948HamletThe Red Shoes, The Treasure of the Sierra Madre(2/4)As IMDB, 2/3 rated
1949All the King's MenThe Heiress, Twelve O'Clock High (2/4)Battleground, The Heiress, A Letter to Three Wives, Twelve O'Clock High (4/4)
1950All About EveSunset Boulevard (1/4)none
1951An American in ParisDecision Before Dawn, A Place in the Sun, A Streetcar Named Desire (3/4)None, though Streetcar ties
In the era of single transferable vote, the Academy picked the later-ranked best film once or twice depending on whether you take IMDB or Rotten Tomatoes ranking; it picked the worst ranked film three times according to Rotten Tomatoes. In the plurality era, it picked the best film once or thrice depending on the ranking and picked the worst film once according to Rotten Tomatoes. Under STV (using IMDB), 35% of nominees score higher than the winning film; that rises to 50% under plurality but the number of nominees drops considerably, and presumably the best nominees are kept. If we use IMDB to place the winning film in the top five nominated films, the winning film's position averages half way between third and fourth place under STV and slightly worse than third place under plurality. Both systems pick a film ranked better than the median of the top four other nominees three times in eight.

So it looks like the Academy picked about as many winners and dogs under both systems last time they ran the experiment and tend towards middling-ranked films. I was kinda hoping that there'd be interesting differences in results to report. There could have been a paper in it had there been. File it under dead ends, and post it so that anybody else that takes an interest can save himself the couple hours. Perhaps there's something in just how bad the Academy's choices are in later retrospect... "The invariance of bad Oscar picks to choice of voting system"...

No comments:

Post a Comment