Beating FiveThirtyEight

Who has the inside track on the playoff? You can ask a number of experts and analysts. One of the more popular computer-based prognosticators, thanks to their affiliation with ESPN, is FiveThirtyEight.

And, so, it seemed like a good measuring stick with which to compare the accuracy of this college football blog. Five weeks before the end of the season we crafted a system of comparison to judge which system was more accurate. Final Score:

CFBComputer: 7
FiveThirtyEight: 4

Analyzing ‘only’ conference champions and playoff participants is, admittedly, a rather narrow comparison. Analyzing only a single year’s data is also narrow. But we’ll chalk up a victory for this site and below give you a breakdown of predictions.

Championship 538 CFBC
SEC-Alabama 75.2 74.2
SEC-Florida 13.4 17
ACC-Clemson 76.8 78.6
ACC-Virginia Tech 16.8 15.2
Big Ten-Penn State 19.2 24.2
Big Ten-Wisconsin 30.8 33.8
Pac 12-Washington 45.6 44
Pac 12-Colorado 29 25.4
Big 12-Oklahoma 64 50.6
Big 12-OK State 22.2 32.8
National-Alabama 16.2 17.6
National-Clemson 34.4 37.8

These numbers represent the average over the last 5 weeks that a given team would win the listed “Championship”. Viewing only the bolded Champions reveals a 3:3 tie between the two computers. Getting into these championship games, however, was predicted slightly better by the CFB Computer, which more heavily favored 4 of the 6 runner-ups.

Except for the National Championship, “winning” any of the above championships is accomplished solely on the field. Being able to predict the slightly subjective choices of the playoff committee is a degree more difficult.

 Playoff Berth 538 CFBC
Alabama 85.8 93.6
Clemson 74.4 81.4
Ohio State 54.6 63.6
Washington 45 39
Penn State 8.6 13.6
Oklahoma 16.2 1.8
Michigan 37.2 37.2

The CFB Computer scores a 3:1 victory here (4:1 when including median), and by a healthy margin. What led to this victory? Two things. One, you’ll notice how much expectation 538 pinned on Oklahoma, a 2-loss team from a middling conference who already lost to Ohio State. Our computer had a similar, though smaller, love affair with Western Michigan, but Oklahoma was well understood to be behind any 2-loss Big Ten Champ.

Two, the CFB Computer weights SOS a little heavier than 538, compared to the number of losses. Fourth seed Washington, the only playoff team 538 weighted more heavily, arguably got into the playoff solely on their 12-1 record (see 2014 Florida State jumping TCU), rather than because of their performance. They were this year’s Cinderella. We simply had higher expectations that they lose another game, and while a Cinderella can monkey with an otherwise perfect bracket, it does not detract from the CFB Computer’s victory over FiveThirtyEight and ESPN’s FPI.

See you next year! Hopefully then we can compare more than 2 systems.


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s