Irrational Expectations in Cricket Redux

This post is in part a follow-up post to this one from 2012 about irrational expectations in cricket, but is more a response to some recent twitter activity in the U.K. BskyB have been using WASP in their coverage of the recent ODI and 20-20 series between England and Sri Lanka, and this has provoked some angry twitter comments. Defenders like David Lloyd 

Folk who understand cricket don't understand WASP. Folk who don't understand cricket understand WASP .. "Who's winning?"
— David 'Bumble' Lloyd (@BumbleCricket) May 28, 2014

or Adam Lewis in this post, point out that a metric like WASP can be very useful for newcomers to watching cricket to give a sense of who is winning at any particular time and how comprehensively. The idea behind Adam’s post is that WASP tells cricket newcomers what experienced watchers already know in their gut. But just how good is the gut of experienced watchers? Well that is hard to measure, but I think it is reasonable to assume that highly paid captains of international teams probably have at least as good an intuition from the game from being actively involved. So let’s look at a very simple decision that captains have to make: whether to bat first or second on winning the toss.

I am currently working on a project with a student from India, Pranav Bhargava, to estimate rankings of teams. In the process we came across the following interesting result: A model that estimates the probability that the team batting second would win an ODI as a function of the quality of the two teams playing, fits the data better than one that estimates the probabiliyt that the team who wins the toss wins the game. Looking at the raw data, we find that the team batting second won 53% of the 1294 games played between May 2002 and May 2014, but the team winning the toss won only 51%. This is a small difference but it is masked the fact that the best team over this period, Australia, batted first more often. When controlling for team ability, the difference is more marked.

This makes no sense at all. While the team batting second wins slightly more often than the team batting first, indicating a second-innings advantage on average, the advantage will not apply in every game, depending on the pitch and the abilities of the teams playing. The captain who wins the toss has the option of choosing to always bat second, or to choose to bat first if these game-specific factors suggest that would be better. Accordingly, the team winning the toss should win more often than the team batting second.

O.K. so let’s give the captains the benefit of the doubt. It seems unlikely with such a large sample, but maybe the random toss has, by chance, been won by the weaker team more often than the stronger team. So we investigated this further. We measired separate team ability measures for each of the top 11 countries (the top 8 + Bangladesh, Zimbabwe, and Ireland) for when they won the toss and lost the toss, and found that for some matchups, losing the toss would be preferable to winning it! In particular, three teams—Australia, Pakistan, and Zimbabwe—make the wrong decision according to the data more than 50% of the time, and so would prefer to lose the toss if playing against a clone of themselves. The remaining teams make the right decision more than 50% of the time, but most are sufficiently imperfect that if playing against Australia or Zimbabwe, would be better off losing the toss and relying on the opposition to make the wrong decision! Only Ireland out of the top 11 teams has a decision record that makes it desirable for them to win the toss against any opposition.

So far, these results replicates results in Bhaskar (2007), but with a slightly different method, suggesting that the results are robust. One criticism of both sets of results, however, is that in using the full sample of games to estimate what should be the correct decision, we are using information from matches that would not have been played at the time captains made their decisions. So we divided the data into two eras of 647 matches each. We used the first era to estimate when it would be better to bat first rather than second, and then used this to compare outcomes to predictions in the second era. We find that teams win more than predicted when captains make the right decision and less than predicted when they make the wrong decision. Put another way, the variable on “correct decision”, is strongly and positively significant in a regression modelling the probability of success. And this uses only information on how well teams have played batting first and second in the first 6 years of the data to predict outcomes in the second 6 years. Real-world captains have more up-to-date information about how teams are playing as well as information about ground conditions on the day. 

At this point, I can’t see any comeback. The information available to our model is strictly less than that available to captains, yet our model can outperform international ODI captains quite significantly.

So what is going on? I think there are likely two sources of imperfect understanding by captains at play here. The first is that captains forget that this is a zero-sum game. If you are a team that is better at chasing than setting a score, but are playing against a team that is much better at setting than chasing, the optimal decision is to bat first, holding conditions equal. But teams possibly play to their own strengths rather than also considering their opponents weaknesses. Another possibility, that I suggested in the earlier post, is a misunderstanding of the regression fallacy: on average, the easier the batting conditions, the higher is the first-innings score. And, on average, the higher the first-innings score, the higher is the probability that the team batting first wins the game, since, on average, higher first innings scores indicate a better than average batting performance. But these two facts don’t in themselves imply that the team batting first has a higher chance of winning when batting conditions are easy.

There are other stories one can tell for the source of the errors made by captains, and we are investigating whether we see in the data what the source is. But the bottom line is that careful data analysis with limited information outperforms professional gut opinion with full information, and by a considerable degree!