Football’s Fight Over Analytics Doesn’t Have To Be So Exhausting

The best thing to happen this NFL season is that coaches are going for it on fourth down more than ever before. The worst thing to happen this season is the people who are paid to talk about football on television are whining about it at a similarly extreme rate.

The NFL set a record for the most fourth-down attempts last season (beating the previous record set in 2019) with 658, and a 55-percent conversion rate. This season, going into Sunday's games, there had been 582 attempts with a 52.4 percent success rate. Just five seasons ago, in 2017, there were only 37 successful two-point conversions the entire season. This year, there have already been 62, with a 50.8-percent success rate. It's clear that coaches throughout the league have begun to think differently about how the game is played.

Just as baseball's own analytics revolution sparked a backlash from the oldest and dustiest corners of the sport, so too has football's new attitude towards fourth downs and two-point conversions kicked off a lot of grumbling from various talking heads. The most extreme example of this backlash occurred two weeks ago, on CBS's studio show, where the hosts spent a few minutes picking apart John Harbaugh's decision to go for two against the Browns, down nine points with just under nine minutes left in the game.

Here’s Harbaugh’s explanation: 

Well, that makes a whole lot of sense! Unfortunately, not one of the five men who are paid to understand football could offer that perspective. 

When Harbaugh is in the moment making that decision, relying on what these studio hosts refer to as "analytics," what he's really using is something coaching staffs call "the book," pages of charts listing every kind of scenario. You might even have seen the book on your television. Some coaches have someone on their staff follow them around on the sideline holding all their charts. Some NFL teams put together their own book with their internal analytics department and others outsource to another company, like Championship Analytics, which works for four different NFL teams and around 60 FBS teams. Michael McRoberts, founder of Championship Analytics, hopped on a Zoom with me so that I could actually see these damn analytics with my own eyes.

The book McRoberts makes, which is customized for each team's settings, has 39 different charts for fourth-down scenarios. It has recommendations on when to accept penalties, when to go for two, and thresholds on an area where McRoberts says coaches are most prone to making mistakes: when you’re able to run out the clock at end-of-game situations with various number of timeouts.

Fourth-down and two-point decisions are determined by comparing how the decision will impact the win probability for the game. The models used for this can vary and use factors like weather (sometimes, a team will request a second book to use if it starts raining), quality of kickers/punters/returners, tempo of offense (which estimates the total number of possessions remaining), and expected spread (using Vegas' predictions to set parameters).

So, when coaches like Harbaugh are making decisions in-game, they are getting kick/go recommendations based on a very specific in-game situation (yard line, point differential, time remaining) that has all the above factors already baked into the model. These models are super complex and factor in every variable that can be quantified. McRoberts said he sometimes has teams request an extra book that covers what to do if their starting quarterback gets injured.

The book looks like a kaleidoscope, with all its different little boxes and rainbow of colors, but McRoberts said these decisions aren't that difficult to explain. "You can have like an hour or two conversation with most people, it really isn't all that," he said. "It is pretty straightforward in terms of the logic."

McRoberts sends out weekly analysis of important coaching decisions to his NFL and college football clients, and internally, this season, he and his staff are compiling the corresponding TV commentary on the decisions. He started his business in 2013, and has never had a media client yet.

"I think it is a real missed opportunity by the commentating community and I think it will happen eventually," McRoberts said. "They are kind of mentioning analytics but they aren't doing it in a way that provides any value to the audience in explaining why they made a decision. It's kind of like when we work with our teams, they don't always follow the recommendations that were given, but they at least follow the logic behind it."

After the Chargers lost to the Chiefs on Thursday night last week, Howie Long had his own moment of analytics agonizing. The Chargers went 2-of-5 on fourth down, but only one of those was really a decision worthy of criticism.

The game has changed, and it's incumbent on the people who are paid a lot of money to talk about it on television to change along with it. This doesn't mean that Boomer Esiason, Terry Bradshaw, and Bill Cowher should never offer criticism of a coach's decision, but it does mean that they should at least put in the effort to actually understand the decisions they are criticizing. Words like "analytics" and "the book" can make it feel like the ideas motivating what's happening on the field are too abstract or complicated to ever fully understand, but that's just not true. I'm no math genius, and I would probably melt into a puddle of anxiety if I ever actually had to read the book on the sideline during a two-minute drill, but it only took a brief conversation with McRoberts for me to come away with a good understanding of how and why the game is changing. The discourse around baseball was genuinely poisonous for several years after analytics made their way into the game, and football feels headed towards a similar fate.

The game isn't going back to the way it was, and writers and talking heads are free to criticize its new direction, but they should at least understand the basics of what they are talking about before they start getting all heated on TV. It's these sports yakkers' job to make fans feel more informed about the action they are watching on the screen, but after spending a season watching them moan about something they refuse to engage with in good faith, I can't bring myself to do anything but change the channel as soon as they start talking.