Calibration of Horse Handicapping

The Preakness Stakes will be run this Saturday, representing the second leg of the Triple Crown and following the Kentucky Derby, the most popular (and most bet) horse race out there. In politics, Nate Silver and FiveThirtyEight have recently summarized the "calibration" of their polling models in predicting the winner of each state's primary/caucus this nomination cycle. How do these two topics intersect? I figured I would do the same thing for horse handicapping by calculating the calibration of the pre-race odds for both the Derby and the Preakness.

As Silver describes it, "Calibration works like this: Out of all events that you forecast to have (for example) a 10 percent chance of occurring, they should happen around 10 percent of the time — not much more often but also not much less often." With any betting odds, there is an associated "implied probability" after you take out the "juice" or vig (the house cut). I used this racetrack chart to get the house cut for both Churchill Downs and Pimlico, and then determined the implied probability each horse (that wasn't scratched) had at winning each race in the past 10 years.

This now allows me to calibrate each race for the past decade. Do long shots win more or less often than they're supposed to? What about the favorites?

Range	# of Horses	E[X]	Winners
[0, 0.05)	128	3.40	3
[0.05, 0.1)	46	3.22	1
[0.1, 0.15)	11	1.41	1
[0.15, 0.25)	5	0.87	1
[0.25, 1]	4	1.15	4

The < 5% long shots (i.e. 20-1 or higher) actually have won just about as often as expected. But what really stands out are the favorites: horses with > 25% have gone 4 for 4, and all in the last 4 years (and the favorite came in 2nd in 2012). Betting the favorite to win/place would have been very profitable in the past half decade.

The Derby has already happened though, so let's provide some insight looking forward. Here is the calibration for the Preakness over the same time frame:

Range	# of Horses	E[X]	Winners
[0, 0.05)	56	1.75	1
[0.05, 0.1)	28	1.84	1
[0.1, 0.15)	9	1.09	0
[0.15, 0.25)	8	1.55	3
[0.25, 0.5)	7	2.54	2
[0.5, 1]	2	1.24	2

Notice that the range of values is much larger, as the Preakness generally has less entrants, so the shortest odds generally are even shorter in this race than in the Derby. There isn't as much of an advantage for the heavy favorites, but horses with greater than a 50% shot against the entire rest of the field are 2 for 2. This year that honor goes to Nyquist at 3-5. Additionally, there appears to be value in the 15%-25% range. The only horse that fits this criteria in 2016 is Exaggerator at 3-1 (who finished 2nd to Nyquist in Louisville). Sometimes it pays to go chalk.