Update: I made another attempt at quantifying luck vs skill in fantasy football here
Fantasy football is a game of chance and skill. And while we all remember wins that felt lucky. I wonder whether that actually matters? Is actually better to be lucky than good? And more importantly, do you need to be lucky to win?
This week, I am going to break down 2 different types of luck that we have to deal with in fantasy football and how much each can impact the final standing.
The first is what I will call real-world luck. This means that due to "chance" you got more points from your players than expected to score. We know from a previous post that ESPN's projections are pretty accurate, but they are not perfect. Players often score more than projected and frankly, they often score more than they should have scored. An example of this was Kenyan Drake last week. He was projected for 12.5 points and with less than 2 minutes left in the game, the Cardinals had the ball up 31-10. The game was over. One first down and they will just kneel the game out. Kenyan Drake had had a good game, scoring 15.5 points at that point. Then he broke free, running for 69 yards and a touchdown. That is an extra 12.9 points that surely swung a lot of fantasy matchups. That is luck. He could have been pulled from the game (why risk an injury when you are up 21?), he could have just stopped after he got the first down (and truly sealed the game), but instead, he took it to the house and won his fantasy managers their game.
The second kind of luck is what I will call matchup luck. There is no defense in fantasy football, there is nothing you can do to prevent your weekly opponent from scoring points. Therefore we can only focus on how much our team scores. This can work in our favor, as we all have those wins where our team puts up a paltry 70, only for our opponent to put up 65 and you get the win. Similarly, we all remember the time our team put up the second-highest score of the week, only to lose because we were matched with the top team. That is also luck.
Today I want to look at the role these two types of luck play. Do teams that finish in first place luckier than their opponents?
Luck in the real world
We can quantify luck in the real world as an error from the projected score. This won't be perfect, theoretically, perfect projections would remove any such luck from the game, but it should give us a good sense of if you need to beat your projections to win. Below is data from 2000 public leagues of how a team scored vs their projected score. Each difference is calculated weekly then totaled over the entire season.
Boxplot of total difference from projected over the course of a season, broken out by regular season standing. the red line marks no difference from projections such that above the red line is good luck, below is bad luck. The numbers across the top are the number of teams in each category.
Here we can see that while the data is noisy, the better teams (better standings) do outperform their projections, with the first-place team outperforming their projections more than 75% of the time. From there it is a pretty steady decline in real-world luck, with teams finishing in 4th place matching their projections on average, and teams worse than 4th underperforming their projections.
Now, we can create our own real-world luck. Better projections will outperform ESPN's projections. Theoretically, you could even max this out with perfect projections, removing all real-world luck from your lineup. But for our purposes and the purposes of this post, ESPN's projections offer a reasonable estimation of what should happen, with deviations being due to random chance (luck). And as the data shows, better teams have more real-world luck.
Billing Department
Matchup Luck
We can classify matchup luck by looking at how a team scored relative to the league. A non-lucky win would be one that would have beat most of the other teams in the league, while a lucky win that would not have beat most of the other teams. Mathematically, this can be represented as saying whether a score is above or below the median score of the week. We can plot this over the course of a season and split our graph into 4 sections as shown below.
Scatter plot of a team's score vs their opponent score, normalized to the league median score that week. Circles are wins, X's are losses, with the different shades being different teams. Circles in the bottom left (green) area would be considered lucky wins, as the team would have lost against most other teams that week. X's in the red area would be unlucky losses, as the team would have beaten most other teams that week.
In the graph above, our origin is the median score of any given week, we then subtract that from how each team scored and can plot whether that score (relative to the median) was good enough for a win (circle) or a loss (x). Any score above the median (top half) in a non-lucky world would be a win, while any score below the median (bottom half) would be a loss. However, this is not a non-lucky world. There are X's in the top right (unlucky losses) and circles in the bottom left (lucky wins). Interestingly, in this 4 team league, 5 outcomes were effected by luck out of the 28 matchups (17%). On top of that, 1 team seems to have benefitted from 4 of those 5 lucky wins. We all know that team that just seems to win even though their team stinks (I rode a team like that to a second-place finish 2 years ago). But what about overall. Do teams that finish higher get luckier with matchups during the year?
To test this, we can use our dataset of 2000 public leagues. We calculate whether a win is lucky and whether a loss is unlucky each week (based on team's score was above or below the league median), and count up the number of lucky wins and unlucky losses a team totals over the course of a season. Below we plot those totals vs the team's final standing.
Line plots of lucky win (green) and unlucky loss (red) totals broken by standing. Error bars represent 95% confidence intervals.
Interestingly, both seem to have an upside-down U shape to their curve. This suggests that the top and bottom teams don't benefit from matchup luck, but that the teams in the middle (finishing between 3rd and 8th place) have more games affected by luck (both good and bad). That kinda makes sense as the teams in the middle will not only be more similar to each other but will also often score closer to the league median, making it easier to cross that line. Overall though, this doesn't suggest that you need good matchup luck to succeed.
Now, we can look one step further and try to classify degrees of matchup luck. For instance, if you were the 3rd best scoring team in a 10 team league and got the win, in a non-luck world this is exactly what should have happened. But there is a little bit of matchup luck here. While you were better than most other teams, you were not better than all other teams. We can therefore quantify how much matchup luck you got for your win based on how many teams would have beaten you (in this case 2 other teams). The top-scoring team would get have no luck involved, the 2nd best scoring team would get 1 luck point, 3rd best gets 2, and so on. We can do the reverse for unlucky losses, the lowest-scoring team gets zero because there were no teams they could have beaten. The 2nd lowest gets -1 unluck points, 3rd lowest -2, and so on. Also note that by making the unlucky score negative, we can calculate the total matchup luck a team got over the course of the season by adding together all their luck from wins and unluck from losses. Below I have plotted this vs final standing
Line plot of luck scores broken out by standing. A positive luck score means more good luck than bad luck in matchups. Error bars represent 95% confidence intervals.
Here we do see a difference. The luckier a team is in matchups, the better their seed. The top team has a luck score of about 11 while the bottom team has a luck score of about -8. In a 10 team league, if you scored the median score, our line designating a lucky win or unlucky loss before, that would be good for 5 luck points or -5 unluck points. So we can say that in total, the top team gets about 2 wins due to luck, while the bottom team gets about 2 losses due to luck when totaled across the entire season.
Now it should be noted that this metric is not perfect. Teams only get luck points on wins and unluck points on losses, so better teams (who should win more and finish with a higher standing) will also have more opportunity to earn luck points compared to unlock points and therefore will artificially have more positive luck score. But it does show that just like in the NFL, you do need a little bit of luck to win the league.
Conclusion
In conclusion, we can see that luck does matter. Both matchup luck (who you play) and real-world luck (how your players perform vs projections) do tend to impact where you finish overall. Unfortunately, that does mean we all need a little luck. Luckily, (see what I did there), we don't need a lot of luck. We also may be able to create our own luck, either with better player projections (which would reduce the effect of luck in the real world) or maybe by creating higher variance plays to produce more lucky wins (stay tuned), but for the purposes of this post, let's leave it as we do need some luck to truly be a great team.
Questions? Comments? Let me know at ac@alexcates.com. Want to read more breakdowns like this? sign up for my newsletter. Finally, like what I do? Consider supporting me on buy me a coffee.
Comments