A surprising find

Bill Simmons, of all people, makes the best argument that I've seen against Bill Belichick's fourth down decision:

Given these realities, if you're feeding me "Here's what happened in this situation historically" numbers, shouldn't we be looking at the data for two-point conversions?

After all, this was essentially a two-point pass play. The Patriots went five wide, stuck Tom Brady in the shotgun, shortened the field and tried to find a quick-hit mismatch. Sure sounds like a two-point play. So what's the recent history of teams passing for a two-point conversion on the road? Peter Newmann from ESPN Research crunched those numbers for me.

2009: 9-for-28, .321 (overall); 3-for-10, .300 (road).

2008: 23-for-52, .442 (overall); 13-for-32, .406 (road).

2007: 14-for-38, .368 (overall); 6-for-23, .261 (road).

That's a great way to look at it. I wish I'd thought of it, although I would much rather see the numbers for all two point conversions on the road, not just passing attempts on two point conversions. Bill's rationale for using passing attempts only is decent (the Pats didn't have a running back in the back field), but it's still relevant because the Colts have to make decisions about personnel before they see how the Pats line up. It's a minor gripe though.

I will also note that the two are not strictly identical, since the Pats can theoretically score a touchdown on the fourth down conversion attempt but not on the two point conversion attempt, but the difference in win probability between converting the fourth down and scoring a touchdown is not large. In any case, the two point conversion a good proxy and the numbers are pretty bad. That's a good data point against my argument.

I'd be remiss though if I didn't chastise Bill for this:

But Indy had already started and completed two long touchdown drives in the fourth quarter against a good defense. Had the Patriots punted, Indy would have had to pull off a third long touchdown drive to win the game. I asked Peter Newmann to research the number of times a team started and completed three touchdown drives in the fourth quarter to erase a double-digit deficit and win an NFL game since 2005. Here's how the list looked before that fourth-and-2 call.

The list, naturally, looks really, really bad. Of course, Bill is overlooking a pretty obvious flaw in his analysis: the question should not be:

"How often do teams pull off three touchdown drives in the fourth quarter to erase a double-digit deficit and win an NFL game?"

The question should be:

"How often do teams pull off three touchdown drives in the fourth quarter to erase a double-digit deficit and win an NFL game given that they have already pulled off two touchdown drives in the fourth quarter?"

I'm not gonna argue that drives in the NFL are IID, but there's no way that it's not relevant information that the Colts had already accomplished two thirds of Bill's criteria when the time came to make a decision. This is basically the argument that the odds of a coin coming up heads after it has come up heads twice in a row are only one in eight because the odds of a coin coming heads three times in a row are one in eight. No, the odds of a coin coming up heads three times in a row are only one in eight if we have no other information. If we know it's already come up heads twice in a row, then the odds are one in two. This is basic stuff.

Furthermore, given that pulling off two touchdown drives under the circumstances is (probably) rare, you probably aren't drawing on a huge sample once you pare it down correctly. Hell, you can even make the argument that the fact that the Colts had already done it twice is evidence that they are not more likely to do it a third time, not less.

In any case, the two point conversion proxy analysis is a great idea. I still stand by my original conclusion that Belichick made the right decision, but I am now less confident that I am correct.