(Inside Science) -- "Ooh, that's a bit shy of the first down," says the play-by-play announcer. "Now we'll see the punter." Mathematically inclined viewers then throw up their hands at the risk-averse nature of decision-making in football. They think teams would improve their chance of winning if they were more aggressive.
Such scenes happen just about every game. Faced with a fourth down, teams often punt or attempt a field goal rather than run an offensive play designed to earn a first down and keep the ball moving toward the end zone. Often, the math is not on their side.
"If [teams] want to maximize their chance of winning the game, they should go for it more often, and they should be more aggressive on fourth downs," said Derrick Yam, a mathematics and business major at Skidmore College in Saratoga Springs, New York, who presented research on the topic earlier this year in Atlanta at a joint meeting of two mathematics societies.
What will the Atlanta Falcons and New England Patriots do if they face tough fourth down decisions at this weekend's Super Bowl? Will they make the conventional, risk-averse choices that can reduce the likelihood that their team will win? (To be fair, Patriots head coach Bill Belichick is known for his willingness to be aggressive.)
Belichick and Falcons head coach Dan Quinn will face second-guessing not only from fans, but also from the New York Times' 4th Down Bot -- a digital creation that analyzes every fourth down of every NFL game and posts its decisions real-time on Twitter. The bot first jumped into the fourth-down debate in 2013, but builds from research done by several people, including Brian Burke, a former fighter pilot turned ESPN football analyst.
Most of the time, the bot's decision is based on just two variables, the down and distance. There are many people -- coaches, fans, pundits and others -- who claim this doesn't always lead to the right answer. For example, let's presume that on Sunday the Falcons' offense faces a fourth-and-2 from its own 42-yard line. Will they go for it? Failing to make the first down would give the Patriots the ball in excellent position to score. Even though the Falcons offense is very good, so is the Patriots defense. The Falcons might weigh a collection of several factors, such as weather conditions, injuries, time remaining and many more, in this fourth down decision.
Further muddying the analysis, teams that do go for it are often behind in the game, and therefore probably not quite as good as the other team.
That's why researchers have been divvying up data to better understand the best strategy. Yam started with a database of more than 500,000 regular season plays, and began comparing and matching them to identify the most similar plays to fourth down attempts, using a statistical method called causal inference. The approach helped the researchers get around the biggest problem with data from real fourth down attempts, which is that trailing teams go for it more often, Yam said. For that reason, the unfiltered data is more likely to include teams that are worse than their opponents.
This type of data sorting is important if you want to understand fourth downs, said Daniel Nettleton, a statistician at Iowa State in Ames, who typically studies the statistics of genetics and genomics research. He also worked with one of his students to study fourth down choices and how they relate to the probability that the team will win the game.
"In sports, a lot of what we're working with is not experimental data and so you always have to be cautious about drawing conclusions when you're working with observational data," he said. It would never happen, but an experiment in which you would randomly assign teams to go for it or not, and then gather results would be very revealing, he said. "That way you can actually get at which strategy is better."
Yam's analysis showed that if teams followed the directions of the Times' 4th Down Bot, they'd see a measureable benefit.
"On average, teams that went for it had a change in win probability 2.6 percent greater than teams that did not go for it," he said. A preliminary analysis suggested a net benefit for a hypothetical hyper-aggressive team that never punted nor kicked a field goal.
Yam noted there is a chance there could be a point of diminishing return because defenses might adapt if offenses began going for it more often. One area left out of his study is also the likelihood of offensive and defensive penalties on fourth down and how that would affect the decision making.
Despite what the analysis says, NFL teams rarely go for it on fourth down. The Philadelphia Eagles went for it 27 times on fourth downs in this year's 16 regular season games, the most in the league. The team with the fewest attempts was the Miami Dolphins, with four. The team's director of analytics is in fact Nettleton's former student, Dennis Lock.
As for this year's Super Bowl competitors, Atlanta made eight of 14 regular season attempts and New England, eight of 12. Neither team has attempted to convert a fourth down in the playoffs this year.