Note an updated version of this post compares ESPN to Fantasy Pros and Reddit Consensus Rankings here.
Every Fantasy player likes to think they are an expert, and some actually are. But this belief in our knowledge or our gut feelings leads us to make lineup decisions against ESPN's recommended starting lineup. So today, I wanted to explore whether or not fantasy players are actually smarter than ESPN.
To do this, I pulled data from 500 public leagues from 2019, giving me information on almost 60,000 lineups. We can go through and retrospectively look at how many points the actual starting lineup scored vs the number of points the lineup would have scored if the fantasy manager had followed espn's projections perfectly. We can look at the raw and percent differences between the two outcomes to produce the distribution low:
Distributions of raw and percent differences between the coach's score and ESPN's projected best lineup's score. In both graphs, the inner thick lines represent a box plot with the white dot at the median while the red line represents the point where the coach scored the same as ESPN.
As we can see above, we as a fantasy manager community are not smarter than ESPN. In fact, ESPN's projected lineup out scores the coach's score about 75% of the time. On average ESPN outscored the fantasy manager (mean = 6points, median = 1.5 points), scoring about 95% as much as ESPN's suggested lineup. So at the very least, most managers would be better off if they listened to ESPN.
You may be thinking that this doesn't apply to you. That you are good at fantasy football and know what you are doing and can beat ESPN. And that may very well be true (25% of managers do on any given week). But we can start to explore this by separating out our point difference by final standing. We would expect better fantasy managers, those who can choose the better starting lineups compared to ESPN, would finish with a better season. We can see the boxplots of the point differences below, both on a weekly and season-long basis:
Box plots of the weekly and season-long raw point difference between Coach and ESPN lineups, grouped by standing. half standings represent ties and the horizontal red line is again our point where the coach perfectly matched ESPN's lineup. Across the top is the total number of weekly lineup decisions in the individual boxplot.
Looking at the graph above, a few things jump out. First, there is a lot of variation, lots of managers at all levels do better or worse than ESPN on any given week. Second, it is not like the best managers are doing significantly better than ESPN. The first-place team is only slightly better than ESPN, with lots of managers scoring less than ESPN on a weekly or season-long basis. Finally, is that the mean does seem to drop with a team's final standing, but the effect isn't that big. The lowest weekly median is only 10 points and when compared to a median margin of victory of 25 points across the sample, those 10 points don't mean much.
Billing Department
Difference in Wins
That last point is important because at the end of the day we don't care about how many points our team scored so long as we beat our opponent. We can therefore reframe the analysis not in terms of points but in terms of wins. Specifically, across a season, we can compare how many wins a team finished the season with compared to how many they would have finished with if they had followed ESPN's projected lineup every week. We will again group teams by final standing. This is especially important to ensure some of the drop off isn't do to players dropping out.
Point plot of the difference in season-long win totals between ESPN's lineup and the Coach's lineup. Error bars represent standard deviation, the red line is the point of equal wins between ESPN and the Coach.
Put simply, we are not smarter than ESPN. On average, the ESPN lineup had 0.9 more wins than the coaching lineup (p < 0.001). On top of that, the worse the finish the more a team would have benefitted from following ESPN's suggested lineup. For instance, an average 10th place team cost their team almost 3 wins a season! (Note, I cropped this graph at 10th place because the numbers of lower seeds were too low causing some random variation). Additionally, while the better managers (by final standing) do make better starting lineup decisions, even the first place team only just barely outperformed ESPN's lineup, and anything less than the first place was worse than ESPN on average.
Following ESPN
Now the final question is do we realize this? Do the people who finish in first follow ESPN's lineup advice more?
Point plot between final standing and the percent of matchups the coach perfectly followed ESPN's projection. error bars represent standard deviation
It terms out that the team that finishes first does follow ESPN's suggested starting lineup significantly more often (p < 0.001), coming in at about 22% of the time, while a 10th place team followed ESPN about 7% of the time. Again it looks like better teams follow ESPN. However, even at the top teams don't normally always follow ESPN, which may be an opportunity to gain an advantage.
Now a couple of important limitations:
First as previously mentioned, this data is pulled from public leagues which I have no idea of how committed the members are. It is possible that players may drop out as the season goes along and may start players who are injured or on bye (which would obviously be against ESPN's and anyone's suggestions). Hopefully by breaking it out by standing will account for some of that, but it's not guaranteed.
Second, it may be that better teams have easier lineup decisions while worse teams are debating bad options. This gives more opportunities to go against ESPN and sometimes there are good reasons for doing that even though it doesn't work (think playing a boom-bust player projected for 7 who could score 0 or 15 instead of a safe player projected for 8 who won't score more than 10). Hopefully, the sample size is large enough that any such moves fall below the noise line, but I don't know.
Conclusion
At the end of the day though, the data suggests you should just follow ESPN's projections and spend your time and mental energy elsewhere. Maybe on finding the right waiver wire adds or trying to get that one trade to go through. Or maybe you can beat the odds and can best ESPN? For me, I am playing the averages and gonna trust ESPN... at least until the playoffs.
Questions? Comments? Let me know at ac@alexcates.com. Want to read more breakdowns like this? sign up for my newsletter. Finally, like what I do? Consider supporting me on buy me a coffee.