Untangling skill and luck on match outcomes

In an interesting blogpost, Patrick A. Ward (Optimum Sports Performance LLC) calculates the contribution that luck has on winning in a variety of professional sports. He uses a methodology described by Michael Mauboussin in his book, The Success Equation, to untangle the contribution of team skill and luck on match outcomes.

Patrick writes:

On page 78, Mauboussin provides the following equation:

Skill = Observed Outcome – Luck

From there, he explains the steps for determining the contribution of luck to winning in a variety of team sports. Basically, the amount of luck plays is represented as the ratio of the variance of luck to the variance of observed outcomes.

Patrick also provides some handy R code that performs the required calculations, where the contribution of luck is given by

$$ \text{Luck Contribution} = \frac{\hat\sigma^2_{\text{Luck}}}{\hat\sigma^2_{\text{Obserbed}}} $$ where $\hat\sigma^2_{\text{Obserbed}}$ is the observed variance in the win % of all the teams, and $\hat\sigma^2_{\text{Luck}}$ is the variance of win % of all the teams assuming each team wins with probability equal to the leagues average win rate (typically close to 50%).

The key takeaway is that if skill plays a much larger role than luck, then better teams will win more often than worse teams and there will be more variance in the observed win % across the league, resulting in a larger $\hat\sigma^2_{\text{Obserbed}}$ and thus smaller ‘Luck Contribution’.

How much luck is there in AFL?

Using the excellent fitzRoy R package, we obtained AFL match outcomes since 1980 and calculated historical Luck contributions.

The plot below shows the historical Luck Contribution % over a variety of rolling periods, 1, 3, 5 and 10 years.

Looking at the blue line for example, we can see that the 3 year luck contribution calculated as of the year 2000 is roughly 20%, looking at the purple line, we can see that the role of luck over any 10 year period seemed to have peaked in ~2006.

The large spike in the 1 year average in 1997 is probably due to the merger of Fitzroy and Brisbane. This would have evened the field a bit, resulting in a reduced values of $\hat\sigma^2_{\text{Obserbed}}$.

AFL match outcomes are very skill based

The table below provides the Luck Contribution % calculated by Patrick over 2019 to 2021 (inclusive) and Mauboussin over 2007 to 2011 (inclusive) for a verity of sports:

Based on our calculations for the AFL over the same time periods, we obtain:

We conclude that AFL is very skill based relative to other professional sports and that the competitiveness of the different sports must fluctuate somewhat in time as there can be considerable differences in the calculated luck contribution % depending on when and over what period it is calculated.