Social Choice and the Oscars

Colby Cosh reports that the Academy of Motion Picture Arts and Sciences is switching from a first past the post ballot for best picture to single transferable vote. I like this, mostly because I go over that system in my public choice class and sometimes get asked if anybody in the real world uses the system. Now I'll be able to say "Well, the Oscars". That's more fun than saying "Well, the Australians".

Voters will rank order the ten nominated films. If any film has a majority of voters picking it as top choice, it's elected. If not, the film with the fewest first place votes is eliminated and that film's supporters are counted as voting for their second choice. Repeat until some film has majority support. Writes Cosh:

Even in the old five-nominee Oscar races, a winner in the single round of voting could conceivably win even though 25 or 30 per cent of the voters loved it and everybody else hated it. (This may have enabled some Best Picture wins later considered controversial, like that of Crash in 2005 or Shakespeare in Love in 1998.) In a 10-nominee field, a mere 15 per cent might be more than enough. The board of governors, mindful that preferential voting was used in 10-nominee Best Picture races from 1936 to 1943, decided to re-adopt the old prophylactic measure against unpopular winners.

It's possible that the old system was more manipulable: if you could get about a fifth of voters to agree on a common first choice, you could potentially get it through despite a majority hating the film. That doesn't worry me too much: I liked both Crash and Shakespeare in Love. What worries me more in the old system is a plurality of uncoordinated voters with mediocre tastes coordinating around some unimaginative Schelling point (Avatar!). So I'll expect fewer oddball good picks like Crash but also fewer lukewarm picks.

I am rather curious about Colby noting the change in voting system at 1943. It would be really neat to compare current evaluations of Oscar winners for the period before and after the change as a measure of which system did better in picking the films later viewed as being very good. Let's give it a shot. Wikipedia lists all the winners and nominees; IMDB and Rotten Tomatoes score the films. A quick flip through doesn't show a clear winner across the two systems, mostly because both fare horribly. I apologize for the table formatting below.

Year	Winner	Bested by (IMDB)	Bested by (Rotten Tomatoes)
1936	The Great Ziegfeld	Dodsworth, Libeled Lady, Mr. Deeds Goes to Town, San Francisco, The Story of Louis Pasteur, A Tale of Two Cities (6 of 9 other nominees)	Dodsworth, Libeled Lady, Mr. Deeds Goes to Town, Romeo and Juliet, San Francisco, A Tale of Two Cities (all rated nominees)
1937	The Life of Emile Zola	The Awful Truth, Captains Courageous, Dead End, The Good Earth, Lost Horizon, Stage Door, A Star is Born (7/9 other nominees)	As IMDB, 7/7 rated nominees
1938	You Can't Take It With You	The Adventures of Robin Hood (1/9 other nominees)	The Adventures of Robin Hood, Boys Town, Four Daughters, Grand Illusion, Jezebel (5/6 rated nominees)
1939	Gone With The Wind	Mr. Smith goes to Washington, The Wizard of Oz (2/9)	Ninotchka, Of Mice and Men, Stagecoach, The Wizard of Oz, Wuthering Heights (5/9)
1940	Rebecca	none	none, though many tie
1941	How Green Was My Valley	Citizen Kane, The Little Foxes, The Maltese Falcon, Sergeant York (4/9)	Citizen Kane, Here Comes Mr. Jordan, The Little Foxes, The Maltese Falcon, Suspicion (5/6 rated)
1942	Mrs. Miniver	Kings Row, The Magnificent Ambersons, Random Harvest, The Talk of the Town, Yankee Doodle Dandy (5/9)	As IMDB, and also 49th Parallel, The Pied Piper, The Pride of the Yankees, Wake Island (9/9: Every other film rates higher on Rotten Tomatoes)
1943	Casablanca	none	The More the Merrier (1/6 rated)
1944	Going My Way	Double Indemnity, Gaslight, Since You Went Away (3/4)	Double Indemnity, Gaslight (2/3 rated)
1945	The Lost Weekend	none	none
1946	The Best Years Of Our Lives	It's a Wonderful Life (1/4)	Henry V, The Yearling (2/4)
1947	Gentleman's Agreement	The Bishop's Wife, Crossfire, Great Expectations, Miracle on 34th Street (4/4)	Great Expectations, Miracle on 34th Street (2/4)
1948	Hamlet	The Red Shoes, The Treasure of the Sierra Madre(2/4)	As IMDB, 2/3 rated
1949	All the King's Men	The Heiress, Twelve O'Clock High (2/4)	Battleground, The Heiress, A Letter to Three Wives, Twelve O'Clock High (4/4)
1950	All About Eve	Sunset Boulevard (1/4)	none
1951	An American in Paris	Decision Before Dawn, A Place in the Sun, A Streetcar Named Desire (3/4)	None, though Streetcar ties

In the era of single transferable vote, the Academy picked the later-ranked best film once or twice depending on whether you take IMDB or Rotten Tomatoes ranking; it picked the worst ranked film three times according to Rotten Tomatoes. In the plurality era, it picked the best film once or thrice depending on the ranking and picked the worst film once according to Rotten Tomatoes. Under STV (using IMDB), 35% of nominees score higher than the winning film; that rises to 50% under plurality but the number of nominees drops considerably, and presumably the best nominees are kept. If we use IMDB to place the winning film in the top five nominated films, the winning film's position averages half way between third and fourth place under STV and slightly worse than third place under plurality. Both systems pick a film ranked better than the median of the top four other nominees three times in eight.

So it looks like the Academy picked about as many winners and dogs under both systems last time they ran the experiment and tend towards middling-ranked films. I was kinda hoping that there'd be interesting differences in results to report. There could have been a paper in it had there been. File it under dead ends, and post it so that anybody else that takes an interest can save himself the couple hours. Perhaps there's something in just how bad the Academy's choices are in later retrospect... "The invariance of bad Oscar picks to choice of voting system"...