This post is written primarily for those cricket fans coming to Offsetting as a result of the WASP being used on BSkyB in Britain to cover the current series between England and Sri Lanka. As with the coverage in New Zealand, it has generated some reaction on Twitter. The aim of this post is to enhance the discussion by addressing some common misconceptions.
What does WASP stand for?
Winning And Score Predictor
How did WASP come about?
WASP was developed by Dr Scott Brooker and his Ph.D.
supervisor Dr Seamus Hogan while Scott was studying for an Economics Ph.D. at
the University of Canterbury.
What forms of cricket is it applicable to?
Limited Overs Cricket (including One-Day and T20 matches)
What does the WASP number mean in the first innings?
The score that the batting team will end up with at the end
of their innings if both teams play averagely from this point on?
What do you mean by "play averagely"?
The model is based on the average performance of a top-eight
batting team against a top-eight bowling/fielding team.
What does the WASP percentage mean in the second innings?
The probability that the team batting second will win the
match, assuming that an average top-eight batting team is playing against an
average top-eight bowling/fielding team.
What does the Par Score mean?
This is a measure of the ground conditions (including pitch,
outfield, overhead conditions and boundary size) that exist on the day of the
match. It is the average number of runs that the average top-eight team batting
line-up would score in the first innings when batting against the average top-eight bowling/fielding
line-up. It is decided by expert opinion (often the commentary team or the
statistician looking at the history of the ground).
Why have a Par Score?
The average score in an ODI is around 250. Without a Par
Score, WASP would predict 250 at the start of the first innings every time.
With the Par Score, WASP may start off predicting 200 if batting conditions are
very difficult or 300 if they are very easy. The Par Score method is not
perfect as it is subjective, but it is a big improvement on simply assuming
that all games are played in the same conditions.
Why does WASP change almost every ball?
That's exactly what it is designed to do. Every ball, whether
the outcome is a six, a dot or a wicket, affects the likely outcome of the
innings and WASP adjusts every time it receives new information with each ball.
If WASP keeps changing its mind, what is the point of it?
It is a measure of how the game is going and allows the
viewer to follow the change in projected final total or probability of winning
as the performance of the two teams ebbs and flows. It is a measure of which team is winning and by how much at any point of time. It also provides a way of assessing whether a partnership that is progressing slowly without undue risk is contributing to its team's probability of success (by steadily increasing the WASP number) or simply creating pressure for later batsmen.
WASP predicted that my team had just a 1% chance of
winning at one stage. They won. Was WASP wrong?
Probably not. We cannot tell by simply looking at a single
match. A 1% chance means that WASP expects that the chasing team will win from
that position one time in a hundred attempts. You may have just witnessed that
one time. If teams regularly come back from a 1% chance of winning to win the
match, then we would be worried about the prediction, but not in the occasional
match. In any sport, teams very occasionally make a miraculous comeback and win from a seemingly impossible position. What this means is that they won from a situation where their probability of winning was extremely low. It does happen, but not very often, which is what makes those games to memorable.
Does WASP consider the skill levels or form of the
individual players or teams?
No. WASP predicts based on the average top-eight team
against the average top-eight team. If the number-one-ranked team is batting
against an associate nation, it is likely that they will do quite a bit better
than the WASP projection. The same applies to a player in a very good or very
poor run of form. Consider the WASP as a benchmark or starting point, and then
adjust up or down based on your view of the relative strengths of the teams and
players.
Because WASP doesn't take into account the particular players, it provides a measure of how well particular players are doing. For example, imagine that WASP was sitting at 225 in the first innings when a partnership between, say, Sangakkara and Matthews begins, and has risen to 300 by the time the partnership ends. That doesn't mean that WASP was wrong; rather it is a measure of how well those two batsmen have played (or alternatively, how poorly the opposition team have bowled and fielded).
Because WASP doesn't take into account the particular players, it provides a measure of how well particular players are doing. For example, imagine that WASP was sitting at 225 in the first innings when a partnership between, say, Sangakkara and Matthews begins, and has risen to 300 by the time the partnership ends. That doesn't mean that WASP was wrong; rather it is a measure of how well those two batsmen have played (or alternatively, how poorly the opposition team have bowled and fielded).
So WASP thinks that all players in top-eight teams are
the same?
No. WASP knows that opening batsmen are on average different
to number 11 batsmen. But it doesn't know that Chris Gayle is not the same kind
of opening batsman as Alistair Cook.
Why does the WASP probability differ from bookmaker odds?
There are a variety of reasons for this. The relative strengths
of the teams, the subjective opinions of the bookmakers and the balance of
their book could all play a role. WASP is a measure of who is winning rather
than who is more likely to win. For example, if Australia is playing Ireland,
and Ireland gets off to a great start, then Ireland are winning at that time
but Australia will still be more likely to win.
Then what is the point of it?
There are a number of sports where a snapshot of the scores at a single point in time does not provide a good indication of who has performed better, and so it is customary to provide more information. In tennis, it is not enough to report that a player is leading 4-3 in a set; we also need to know if the games have gone with serve or the if the player who has won 4 games is up a break. In golf, the total number of strokes is never reported for players who haven't finished the round; instead, score updates report the players' deviation from par. That essentially assumes that the player will achieve the par (mode) score on each remaining hole. WASP is a similar reporting of deviation from par.
I want to follow how my team is doing; what should I be looking for?
A batting team's likely score in the first innings or their probability of reaching the required target in the second innings always falls when they lose a wicket. This cost is typically much higher at the start of an innings than later on, when wickets in hand may not be so important. To allow for the inevitable fall in WASP when losing a wicket, a batting team should be looking to see WASP steadily increase during a partnership. If the bowling team can keep the increases in WASP low during each partnership and still take regular wickets, it can come out on top in the innings.
Could WASP be an alternative to Duckworth-Lewis for adjusting targets in rain-affected matches.
Yes it could. Sometimes, WASP will give a very different answer, partly because of its using a different criterion for fairness and partly details in the implementation, including it having an adjustment for the ease-of-batting conditions. Most of the time, however, it will produce a similar outcome. For example, in the first ODI between Sri Lanka and England, D/L initially set Sri Lanka a target of 259 as a result of the earlier rain interruption; WASP would have set 262. After the second rain interruption, D/L revised the target to 226; WASP would have set 220.
What if I have more questions?
Yes it could. Sometimes, WASP will give a very different answer, partly because of its using a different criterion for fairness and partly details in the implementation, including it having an adjustment for the ease-of-batting conditions. Most of the time, however, it will produce a similar outcome. For example, in the first ODI between Sri Lanka and England, D/L initially set Sri Lanka a target of 259 as a result of the earlier rain interruption; WASP would have set 262. After the second rain interruption, D/L revised the target to 226; WASP would have set 220.
What if I have more questions?
Comment below, or ask us on twitter @srbrooker and
@seamus_hogan.
Reading this as it links off a recent post. This is excellent, as usual.
ReplyDeleteExcellent stuff. Many thanks
ReplyDeleteWith the 1% example, this is where I become skeptical. Surely this model assumes some underlying mathematical distribution that may or may not reflect reality (likely to be the latter). There is essentially no way to really 'test' this system given that you can't replay the exact same game with the exact same parameters.
ReplyDeleteSo (for example) isn't the number of predicted runs an example of false precision, in that to be truly meaningful you would have to at least give some additional parameters such as the number of runs +/- a distribution parameter.
Thanks for your comments V - I think you raise two good points. I agree that you can't test the system by replaying the exact same game. Indeed any individual game situation occur incredibly rarely. However, you can look at a large number of games where the probably of winning was (say) 1%, even if that 1% was a result of different game situations (eg. 30 runs needed from the last over with 5 wickets in hand vs 50 needed from 20 overs with the last pair at the crease). And then you can assess whether, on average, the batting team ended up winning 1% of those matches. On your second point, yes, in the first innings we are predicting the mean outcome. It would be great to assess the projected scores vs the actual outcomes to show the likely degree of variance from the mean. We will make the effort to look more closely at this when we get the chance.
ReplyDeleteFollowing on from Scott's reply, there are two sources of uncertainty. First, the prediction (which we would rather call a projection), is just the mid point in a sense of a range of things that could happen. We like to think of it as a measure of how a team has performed so far rather than a statement of what will happen. Second, though, there is model uncertainty, in which we could have mis-estimated the expected future score or probability of winning, maybe because of evolving strategies, changes to the rules and so on. If so, reporting the model to the nearest integer might look like excessive precision, but reporting to the nearest multiple of 10 in the first innings, say, would mean situations would arise where the WASP projection would keep changing by 10 every ball.
ReplyDeleteThe other thing I would say is that we should never take probabilities (in any environment) too literally. We know that the human brain is not very good at understanding probabilities intuitively. If I see a number like 1%, it doesn't matter if it means that a win from that situation would only occur once every 100 games, literally, or maybe would occur once every 50 (i.e. should be reported as 2%). What matters is that 1% is a very low number and is lower than 2%, so when I see WASP fall from 2% to 1%, I know that the team is almost certainly going to lose, and things are getting worse.
Chris,
ReplyDeleteI've only just seen this comment, so sorry about the delay. The answer is that it does. It possibly takes it too much into account. Before the most recent rule changes, the bpp had added only about 4 runs to teams scores on average. With the new rules that require no more than 4 fielders outside the circle at all times (in contrast to the previous rule of 5), the difference between the normal situation and the bpp (maximum of 3) is smaller than it used to be.