The trouble with Net Run Rate

In any competition in which there is pool or round-robin play to rank teams before playoff rounds, there needs to be some method of deciding the relative ranking of teams who finish equal on wins and losses. Ideally, this method will reward the teams that have performed best, and also not create any perverse incentives for teams to do anything other than act in a way to maximise their probability of winning.

A nice example of perverse incentives came in the 1999 Cricket World Cup. Only two teams out of New Zealand, Australia, and West Indies were going to carry on from their group into the next round. The rules were such that teams carried through only their results against other teams that made it to the next round. Prior to the match between the West Indies and Australia, New Zealand had beaten Australia but had lost to the West Indies. Australia therefore needed to beat the West Indies, but also wanted WI to be the team that carried through with them so that their loss against NZ didn't matter. As is traditional in the Cricket World Cup,the method used to rank teams with equal numbers of wins and losses, was net-run-rate (NRR)--the difference in a team's average runs scored per over faced and its average runs conceded per over bowled. Batting second, Australia therefore did a deliberate go-slow in order to win, with their 5th wicket partnership taking an extraordinary 127 balls to score the 49 remaining runs needed for a win. This was designed to elevate the West Indies' NRR above New Zealand's. As it turned out, the strategy was not successful, as New Zealand still had a match against the lowly ranked Scotland, and took extraordinary risks to not only win that match but win it by a sufficient margin for their NRR to overtake the West Indies'.

In the current World Cup, there isn't the same "super 6" 2nd stage where teams only carry through some of their points from the first round, but NRR is still used as the tie-breaker. This system is still flawed, as exemplified by Tuesday's match between New Zealand and Scotland. Anyone looking at the two innings scored could be mistaken for thinking that the match was close. It wasn't. What happened was that New Zealand bowled Scotland out for a very low total, and was almost guaranteed a win. When it was New Zealand's turn to bat, they strove to win the match in a few overs as possible, in order to maximise their runs-per-over figure. The fact that they lost 7 wickets in the attempt meant that they did present Scotland with the sniff of a chance of an upset, but the 7 wickets will have no bearing on their eventual NRR.

This exemplifies three problems with NRR:

1. The effect of a large win against a lower-ranked team on NRR depends on which team bats first, since the team batting second only bats until it has overtaken the other team's score, meaning that that innings gets a lesser weight in the runs-per-over calculation than an innings where all 50 overs are faced. 
2. The magnitude of a victory when the team batting second wins is a function not only of how many balls it took the team to amass the winning total but also the number of wickets lost in the process. NRR only takes the former into account. This creates the perverse incentive where New Zealand put their win (slightly) at risk by worrying only about how many overs they used and not how many wickets they lost. 
3. The ranking of two or more teams should not depend on which one beat up the most on a team ranked well below them. If, as could easily happen, three teams (say, Australia, New Zealand and Sri Lanka), finish in a tie for first place in their group, the determination on goes through the quarter finals ranked 1st, 2nd, 3rd, should not come down to which team beat Scotland b the biggest margin.

So with these flaws in mind, here is a sequence of proposals to replace NRR with a different tie-breaking rule.

Adjustment 1: To deal with the first problem above, use the average margin of victory/loss rather than NRR: If the team batting second loses, its margin is its score divided by the score required to tie the match. This will be less than 1. The winning team's margin is the reciprocal of this--the target score divided by the chasing team's score. If the team batting second wins, its margin is the number of balls available to it + 1 divided by the number of balls actually used. The losing team's margin is again the reciprocal of this. In the case of a tie, the margin is 1.0 for both teams.

Adjustment 2: To deal with the problem of teams sacrificing wickets for the sake of fast scoring, amend Adjustment 1 in the case where the team batting second wins, by dividing the predicted score at the end of 50 overs by the score required to tie (the implicit score predictor in Duckworth-Lewis would work for this, although I'd prefer to use WASP due to its adjustment to conditions).

Adjustment 3: Make the calculations iteratively. Let there be n teams in a pool. Construct the table at the end of pool play using points scored, and using Adjustments 1 and 2 to rank teams otherwise tied. Then remove the bottom-ranked team and give them a rank of n. Now reconstruct the table using only games played amongst the remaining n-1 teams, and again find the lowest ranked team. Give it rank n-1, remove it and reconstruct the table with the remaining n-2 teams, etc. As an example of how this could be beneficial, imagine that in the current world cup, Sri Lanka beat Australia, Australia beat NZ, and all three beat England and the other three teams except that the game between Australia and Scotland is rained out. Under the system in place for this competition, Sri Lanka and NZ would finish ahead of Australia simply because Australia were denied to opportunity to play Scotland. Under Adjustment 3, the games against Scotland would be irrelevant for deciding the relative ranking of the top three teams. *

Adjustment 4: O.K. now I am getting well out of the realm of feasible rules into the kind of competition we would have if the ICC comprised exclusively economists, but it is fun to speculate. My adjustment 2 still does not properly align incentives because maximising the expected margin of victory is not the same thing as maximising the probability of victory. So instead, let's define the margin of victory in the following way. Draw the WASP-worm graph of the percentage probability of winning for the second innings as a function of the number of balls bowled. This is a graph  is contained within a rectangle that has a length of 300 and a height of 100. The value for the team batting second would be the area under the graph divided by the area above it. The value for the team batting first would be the reciprocal. Using this method, it would be possible for the winning team to have a lower score than the losing team, but no matter: this scheme means that the way to maximise your team's tie-break variable would be to maximise your probability of winning.

Adjustment 4 tries to align incentives with the only thing that should ever matter in sport--trying to win--but it doesn't deal with the situations like Australia's go-slow against the West Indies in 1999 (or NZ's go slow against South Africa three year's later that shut Australia out of their own tri-series final). The format used in this year's World Cup does not contain the possibility of such strange incentives, but Adjustment 3 would add that. With an obvious nod to the Gibbard-Satherwaite Theorem and Arrow's Impossibility Theorem, then, let me suggest throwing all of these out the window and instead using the following manipulation-proof tie-breaking formula:

Adjustment 5: Rank all teams leading into the tournament based on recent performances. In the event of two or more teams being tied on points at the conclusion of pool play, their relative ranking will be according to their pre-tournament ranking, fully independent of play during the tournament.

* The ICC might argue that they have addressed the problem in a simpler way by restricting the next tournament to only 10 teams. But the results to date in this World Cup suggest that there will still be some very weak teams and non-competitive matches given the non-competitive process for selecting the 10 teams.