Analyzing the accuracy of Matthew Berry’s weekly fantasy football “Love/Hate” list for the 2016/2017 NFL season
Written By: Aedan Marty (@FirstAedKit)
During the 2015/2016 NFL season, an estimated 74.7 million Americans spent $4.6 billion playing fantasy football , with both of those figures expected to grow in the coming years as fantasy sports continues its meteoric rise across the U.S. and around the world. ESPN.com is among the most popular sites where fantasy football hopefuls join a league, compete with friends, and attempt to win their league and acquire the accompanying bragging rights. At ESPN, the go-to guy for all things fantasy football is their Senior Fantasy Sports Analyst Matthew Berry, AKA the Talented Mr. Roto.
Every season, millions of fantasy footballers rely on the advice of Mr. Berry to draft their team, set their weekly lineups, and find a waiver-wire gem who will propel them to victory. One of Mr. Berry’s most popular in-season weekly columns is his “Love/Hate” list, where he selects several players at each position (QB, RB, WR, TE) and places them in his “Love” list or his “Hate” list. Players who make the “Love” list are players who Mr. Berry believes are undervalued for the given week, due to a favorable matchup or any event that he believes will increase that player’s fantasy value for that week. Alternatively, players who are relegated to the weekly “Hate” list are players who Mr. Berry sees as overvalued for a given week, due to unfavorable matchups, lingering injuries, or any other reason that might limit their productivity for the week.
As a loyal reader of the “Love/Hate” column for the past several years, there has always been one question on my mind: how talented is the Talented Mr. Roto? Do players on his “Love” list justify their placement with above-average production that week, and do players on the weekly “Hate” list usually underperform for the given week? To answer my question, I looked at ESPN.com fantasy football scoring data across the 2016/2017 NFL season  and compared it to scoring projections that ESPN.com forecasts for NFL players each week during the season. I chose to analyze the accuracy of the “Love/Hate” list on a positional basis to see if particular positions are harder to project than others, and the possible reasons behind this.
For my analysis of player projections and scoring, I chose to use the method I was most familiar with: ESPN.com’s Point-Per-Reception (PPR) decimal scoring, explained below:
To provide additional clarity, made-up stats and points for a QB and RB can be found below:
QB Ben Roethlisberger, PIT: 23-30, 287 passing yards, 2 passing TD, 1 INT, 12 rushing yards
287 passing yards * 0.04 points per yard = 11.48 points
2 passing TD * 4 points per passing TD = 8 point
1 INT * -2 points per INT = -2 points
12 rushing yards * 0.1 points per rushing yard = 1.2 points
Total = 11.48 + 8 – 2 + 1.2 = 18.68 points
RB Le’Veon Bell, PIT: 26 rushes, 134 rushing yards, 1 rushing TD, 5 receptions, 57 rec yards
134 rushing yards * 0.1 points per rushing yard = 13.4 points
1 rushing TD * 6 points per rushing TD = 6 points
5 receptions * 1 point per reception = 5 points
57 receiving yards * 0.1 points per receiving yard = 5.7 points
Total = 13.4 + 6 + 5 + 5.7 = 30.1 points
To answer my question about the accuracy of the “Love/Hate” lists, I created a metric called Points Above Projection (PAP), which measures the difference between a players’ projected scoring total for that week and their actual scoring total. If players score more than his weekly projection, their PAP will be greater than 0; however, if a player fails to meet their weekly projection, his PAP will be negative.
To support the notion that Mr. Berry is as talented as we expect, we would anticipate seeing players on the “Love” list yield PAPs that are positive, indicating they were undervalued compared to their projections, and thus scored more points than projected. The opposite would go for players on the “Hate” list; if Mr. Berry believes players on his “Hate” list are overvalued for that week, we expect to see a majority of these players yield PAPs that are negative, meaning their realized scoring total for the week failed to live up to projections.
The weekly PAP for each position that can be seen in the graphics below are a cumulative average of all the players’ PAP that Mr. Berry selects for his “Love” or “Hate” list each week. Typically, 2-5 players at each position make the “Love” and “Hate” lists every week, so the data points are the average PAP of those 2-5 players, which allows for a more comprehensive analysis of positional accuracy as opposed to individual players that are selected for the “Love/Hate” column every week.
For example, consider quarterbacks A B and C, who are all on the “Hate” list for week X:
Actual Points Scored
In order to find the PAP weekly average for quarterbacks on the “Hate” list for this week, we take the average of +2.1, -1.9, and -2.3, which is -0.7. This says on average, the quarterbacks on the “Hate” list underperformed by 0.7 fantasy points compared to their week X projection, which is what we would expect to see if Mr. Berry is making correct picks. This -0.7 number is what would appear for “Hate” list on week X in the QB graph below. Now on to the data!
Min PAP: -7.17 (Week 3)
Max PAP: 6.28 (Week 5)
PAP Weekly Average: -0.52
Min PAP: -4.53 (Week 7)
Max PAP: 2.83 (Week 10)
PAP Weekly Average: -0.06
The quarterback is generally regarded as the most consistent fantasy football producer in terms of week-to-week scoring. Unlike other positions in fantasy football, such as running back and wide receiver, quarterbacks have the ball in their hands virtually every play, which means more opportunities to make plays and score fantasy football points. Despite this, quarterback performance depends on many variables. In the 2016/2017 NFL season, the Cleveland Browns gave up the most fantasy points per game (FPPG) to quarterbacks, at 19.46 FPPG. Conversely, the Denver Broncos were the stingiest defense to face as a fantasy quarterback, yielding just 11.93 FPPG during the season. This did not seem to be lost on Mr. Berry when selecting quarterbacks for his Love/Hate list throughout the season. Quarterbacks made the “Love” list four times during the week they were playing the Browns, compared to zero quarterbacks who made the “Hate” list when facing the Browns that week. On the contrary, quarterbacks who had the Broncos on their schedule for the week did not make the “Love” list once all season, but quarterbacks made the “Hate” list seven times out of 16 weeks when they had to face the Broncos defense.
By taking a look at the numbers, we can see that quarterbacks who made the “Love” list slightly underperformed compared to their ESPN projections, with the PAP weekly average at -0.52 FPPG during the season. Quarterbacks who made the “Hate” list throughout the season also underperformed on average during the season, although by less than their “Love” list counterparts. The PAP weekly average for quarterbacks on the “Hate” list yielded -0.06 FPPG, with a slightly lower standard deviation of PAP weekly averages than “Love” list quarterbacks. While there were good weeks and poor weeks for PAP weekly averages throughout the season for quarterbacks, the season-long data on quarterbacks reveals that both “Love” and “Hate” list quarterbacks perform at or slightly below their ESPN projections. In this sense, the “Love/Hate” column is a good source of additional insight by ESPN fantasy insiders like Mr. Berry, but the statistics show an important trend. Quarterbacks on the “Love” list do not yield consistently above-expected results, while quarterbacks on the “Hate” do not always flop, and can still have impressive fantasy scoring weeks despite their placement on the “Hate” list.
Min PAP: -8.40 (Week 12)
Max PAP: 8.70 (Week 1)
PAP Weekly Average: 1.12
Min PAP: -12.80 (Week 16)
Max PAP: 13.15 (Week 5)
PAP Weekly Average: 0.66
Running backs are a tricky position to evaluate in this day and age of fantasy football. Compared to a decade ago, the NFL is a much more passing-driven league, evidenced by the uptick in passing statistics and consequential decline in rushing statistics throughout the league during the past several years. As the days of the bruising 240-pound running back (think LeGarrette Blount) become a distant memory for more teams every year, a more complete, pass-catching running back has become more desirable for teams (hello, Le’Veon Bell & David Johnson). This makes projecting fantasy football scoring for running backs increasingly difficult, especially in point-per-reception scoring formats, which was the format used in this study.
Unlike with the quarterback position, both “Love” list running backs and “Hate” list running backs exceeded projections on average throughout the season. Running backs on the “Love” list yielded a PAP weekly average of 1.12 FPPG across the 16 weeks of the fantasy season, and “Hate” list running backs beat their weekly scoring projections during the season, with a PAP weekly average of 0.66 FPPG. While running backs on the “Love” list beat projections by a greater amount than running backs on the “Hate” list in terms of PAP weekly average, this is not necessarily a great result for readers of the “Love/Hate” column. If running backs from both lists are beating weekly projections, it is hard to glean too much information from either list about who to “Love” and who to “Hate” for the week. The positive PAP weekly averages from both lists could be a function of the point-per-reception (PPR) scoring format that rewards running backs for both rushing and receiving stats. Running backs who catch passes out of the backfield add another dimension to their fantasy football scoring potential; they provide most of their scoring through rushing stats, but additional points they earn by catching passes are “icing on the cake” for savvy fantasy owners who prioritize pass-catching running backs in PPR scoring formats like the one used in the study.
Min PAP: -9.90 (Week 15)
Max PAP: 1.40 (Week 1)
PAP Weekly Average: -2.09
Min PAP: -8.45 (Week 4)
Max PAP: 12.70 (Week 10)
PAP Weekly Average: -1.35
As the NFL has gradually shifted to a more passing-oriented style of play calling, wide receivers have seen their fantasy football stock rise dramatically during this span. More passes mean more opportunities for receivers to score fantasy points, which has forced fantasy players to draft receivers earlier than ever before. This year marked the first year ever for ESPN Fantasy Sports that two receivers, Antonio Brown and Odell Beckham Jr., held the top two spots in ADP (Average Draft Position) rankings. On average, Brown (Pick 1.4) and Beckham Jr. (Pick 2.9) were the two players chosen earliest in ESPN fantasy football drafts across all drafts hosted on ESPN.com this season. No longer than a decade ago, it would have be rare to see any receiver crack the top 5 in ADP rankings, let alone hold the top two spots. This speaks to how the pass-heavy revolution in the NFL has increased receiver value while consequently decreasing running back value, who receive less rushes per game and less opportunities to score fantasy points.
Selecting wide receivers for his weekly “Love/Hate” lists appeared to give Mr. Berry the most difficulty throughout the season, compared to other positions. On the “Love” side, Mr. Berry’s picks achieved a PAP weekly average that was greater than zero for only 3 of the 16 weeks during the NFL season. This underwhelming performance by the receivers resulted in a PAP weekly average of -2.09 FPPG across the 16 weeks of the fantasy season. For receivers who made the “Hate” list during the season, their performances also failed to meet projections (which is what we expect to from the “Hate” list players), resulting in a PAP weekly average of -1.35 FPPG during the 2016 fantasy season. While this underperformance by receivers on the “Hate” list would seem to indicate good selections made by Mr. Berry, there are several reasons why this is misleading. The first reason is due to the season-long underperformance of receivers on the “Love” list as well. In contrast to the situation with running backs, since both “Love” and “Hate” list receivers underperformed throughout the season, it is difficult to laud Mr. Berry for his “Hate” list picks, when the “Love” list picks actually performed worse on average throughout the season. The second reason is that, as the chart shows, some of the highest weekly PAP scores actually came from “Hate” list receivers, whereas “Love” list receivers failed to achieve a weekly PAP score higher than +1.4 FPPG, which occurred during the first week of the season.
Min PAP: -6.15 (Week 1)
Max PAP: 5.25 (Week 15)
PAP Weekly Average: -1.27
Min PAP: -4.50 (Week 15)
Max PAP: 7.20 (Week 2)
PAP Weekly Average: 0.75
The tight end position in fantasy football is a bit like picking the last available kid in gym class to join your kickball team: you do not expect much production from them, and any above-average results from them are a welcomed sight for your team. Unless you have one of the premier options at tight end, like Rob Gronkowski or Greg Olsen, the weekly performance for the position is very volatile and often heavily dependent on a red-zone touchdown catch to yield a “good” scoring week from a tight end. Because of their larger build and lack of explosiveness (relative to receivers and running backs), tight ends are mostly used for short to intermediate passes and even blocking in some cases. This limits their fantasy scoring potential, which sometimes causes them to be an afterthought for fantasy players who look to draft players with a higher scoring potential on a weekly basis.
Because of their limited scoring appeal, Mr. Berry often selected fewer tight ends to make his weekly “Love/Hate” list, choosing between 1-3 for each list instead of the normal 3-5 players he selected at other positions. Like with receivers, Mr. Berry struggled to provide impressive selections on a weekly basis. Tight ends who made the “Love” list during the season underperformed on average, yielding a weekly PAP average of -1.27 FPPG across the 16-week season. On the contrary, tight ends on the “Hate” list over performed on average, producing a weekly PAP average of 0.75 FPPG, slightly beating projections during the course of the season.
This article examined the week-to-week accuracy of Matthew Berry’s “Love/Hate” column, and the performance of players selected by Mr. Berry relative to their ESPN fantasy football scoring projections. The season-long positional data helps quantify the value of using the “Love/Hate” column to help you select weekly lineups. There were weeks that helped make Mr. Berry’s case as truly being the “Talented Mr. Roto,” but other weeks where Mr. Berry’s picks fell flat. As a whole, it is difficult to give or take away credit from Mr. Berry, because of all the variables that go into fantasy football performance on a weekly basis, including injuries, weather, and matchups. If anything, this article should be fair warning that the fantasy football experts employed by ESPN and other companies are no more informed than the everyday fantasy football player. Their articles should not be taken as sacred scripture that will lead you to the fantasy championship if you follow their advice, but rather should be viewed as an opinionated article written by knowledgeable authors to provide another perspective to fantasy players when setting their lineups next season.