Those that have played Fantasy Football for a long time will know this feeling. You pay attention to the waiver wire every day. You churn the back of your bench, looking for the next player that can break out for you, only to have that player you dropped 2 weeks ago go off when your opponent starts them against you. How could you have dropped them, see how good they are? When this happens, you may wonder, is your roster churning worth it? Or would you be better off trusting the players you selected, making a move here or there but generally leaving your roster alone and giving them the time to break out?
Today we are going to try to answer this. By looking across thousands of teams and leagues, we can look at if it is better to churn your team or trust your players. Should your starters stay starters, or should you swap out players to play the matchup?
The Data
We will be looking at 2 different actions made in ESPN Leagues, specifically the number of transactions made during a week (this includes adds, drops, and trades) and whether they changed their starting lineup (does their starters this week match those of the previous week?). Unfortunately, because of limitations with the ESPN API, we can only get transaction data for a subset of leagues. In total, we will be looking at transaction data from ~4000 teams and roster changes from ~74,000 teams over the last 3 years. This should give us plenty of data to see what trends we can find.
(Note, if you want to support this work or contribute to the data, the easiest way is to sign up your ESPN league at fantasyleaguereport.com)
Engagement Trends
Before we look into how our moves affect performance, let's look at whether there are trends in how many moves managers make across the season. There is a narrative that people drop off and stop paying attention as the season progresses, but is that true?
Graphs of the percentage of teams that changed their starting lineup (left) or made a roster move (right) per week. Different lines represent different seasons (2019, 2020, and 2021)
There is an interesting trend here, even if it is not that surprising when you think about it. First, both graphs show an increase in engagement as the season progresses before taking a dive at the end of the season. This is directly counter to the prevailing consensus that players drop off across the season. Instead, I think this can be understood when we look at the specific weeks. Bye weeks don't start until week 4, so teams may be more inclined to keep their lineup set until they have to change it because of bye weeks. Similarly, we see the percent of teams that make a transaction follow a similar pattern (though less of an increase). I also chalk up the drop off at around week 13 as just being teams that are eliminated from the playoffs and so are essentially done playing. Both graphs support the idea that engagement doesn't change that much throughout the season, having more to do with the demands of the fantasy season than any change in interest. But what about success, do these moves help?
Should You Change Your Starting Lineup?
Week to week impact
Line plots of the odds of winning your matchup on a given week based on if you changed your starting lineup (orange) or not (blue) from the previous week's lineup. Shaded areas represent 95% confidence intervals.
Unsurprisingly we see that altering your starting lineup is kind of table stakes for competing. We see that teams that do change their lineup win about 50% of the time, which is chance in this case. Instead, it is the managers that don't change their starting lineup who tend to lose more often. This is particularly true once the bye week start as these managers may be leaving players on bye in their starting lineup. Beyond that I don't see much variation or pattern for a given week. Next let's look across the season.
Season Long Impact
The total wins a team earns vs the percentage of weeks where they changed their starting lineup. Line and shaded area represent the line of best fit. the points are binned with 95% error bars with each representing the same number of data points.
Here is probably the biggest impact we will see today and frankly the most interesting. First, we see that if keep your starting lineup the same the entire year then you are going to have a rough season, no surprises there. Interestingly, there is little difference between a team that changes their lineup 20% of the time and 60% of the time, though more is better (gaining about 1 third of a win on average). However, once you get above there, there are real returns going all the way up to teams that are constantly changing their lineup winning almost 2 more wins over the course of a season than those that only change 60% of the time.
My takeaways from this graph are twofold. First, is that you need to be changing your lineup sometimes, that is table stakes. This makes sense with things like bye weeks and injuries requiring you to make some amount of lineup change. I think the other end is more interesting. Teams that change their lineup constantly get a real advantage. I wonder if this suggests streaming a position is beneficial. That may be something to look at in a future post.
Now that we have established that changing your starting lineup is generally a good thing, what about the rest of your roster?
Do Roster Moves Help?
Week to Week
Line plots of the win rate (odds of winning in a given week) based on if you changed your roster that week. shaded areas represent 95% confidence intervals.
When we split teams by if they made a move or not, we see an immediate (if slight) advantage form. Teams that made a move won 1-2% more often than random chance and with 3-4% better odds than teams that did not make a move. Interestingly, there is a spike at the end of the season, with teams that make a move winning over 55% of the time, though this change is not mirrored in teams that did not make a move. My guess is that something to do with playoffs, teams being eliminated, and managers more willing to move for 1- or 2-week players at the end of the year may explain this bump at the end. Maybe some owners keep trying to win while others stop once they are eliminated, or maybe moves during the playoffs are key to success. I am not sure, but at a bare minimum, making a roster move does seem to help on a week to week basis.
What about across the season?
Linear regression of the percent of weeks a team made a roster move during the season and their total number of wins that season. Scatter points are binned into groups of the same number of underlying data points, shading represents 95% confidence interval.
Again, we see that making roster moves has a small but positive impact on our season long win totals. Basically making at least 1 move per week adds up to about 1 additional win across the season. That is a small but meaningful bump. That said, there is also a lot of noise here so I wouldn't take this as evidence that you should make a move just to do so, but the bias should be towards adding players and changing your roster.
What about the number of moves?
Box plots of the number of weekly win rate (odds of winning) split by the number of moves a team makes. Horizontal line is at the base rate of 50% chance of winning.
Here we see two interesting trends. First, as we saw above, in general you want to be making at least 1 transaction per week, with even up to 3 or 4 seeming to increase your odds of winning that week. Second, while the trend seems to generally continue, it also gets a lot more variable as the number of transactions increases (we can see the size of our box in the box plot increasing). This is likely just a factor of the number of samples with 18000 teams making no moves, ~10,000 teams making 1 and 2 moves, while only 500 making 8 transactions in a given week. That said, another explanation could be that there are 2 types of teams that make a lot of transactions. Some of these teams are top heavy with really good starters but no reliable bench players so they churn their bench constantly. These teams are actually a good bet to win a given week (high win rate) but risk injuries ruining their season at some point. The others are just bad teams throwing players at the wall and hoping to find some that stick. These teams are bad and unlikely to win in a given week (low win rate). That combination would lead to more volatility. I will admit that I do not see any obvious signs that this dichotomy is to blame for the variability with more transactions, but I am not ready to rule it out.
Linear regression of the average number of transactions per week vs the total wins a team gets. Scatter points are binned into groups of the same number of underlying datapoints.
When we combine the total wins across the season and compare it the average number of transactions per week, we see that teams that average more transactions per week (in this case just the same as saying teams that make more transactions) win more games. Again, the effect is small, probably accounting for half a win across the entire season and it is also frankly noisy. But again, we find that our bias should be towards making a roster move.
Limitations
While this dataset is not all public leagues anymore (with the transaction data being exclusively private leagues), it still should be mentioned that I am not making any judgement of manager or league competitiveness. Additionally, I am not breaking out what type of move was made or trying to determine why a lineup change was made (for instance was a player on bye or did the manager actually think the swapped player would perform better). I think these questions may be fuel for future blog posts so stay tuned to see if we can't try to better understand the impact of different types of roster moves.
Conclusions
In conclusion, I would generally say that activity is a good thing and that more of it will help your team. We saw a pretty large impact of changing your starting lineup each week with some additional small bumps gained from changing your roster each week. In future posts we can dive into what types of roster moves are the best and maybe try to look at whether streaming is actually worth it (not just if it is possible to stream a good qb, but if the average manager can do it.
Questions? Comments? Let me know at ac@alexcates.com. Want to read more breakdowns like this? sign up for my newsletter. Finally, like what I do? Consider supporting me on buy me a coffee or by signing up for fantasyleaguereport.com.