I kind of buried the post, but I wanted to do a follow up on my post arguing that the Oilers current hot streak (and implicitly, their somewhat respectable looking record) is a mirage, I took a look at some of the shootout numbers. Frequent Oilogosphere commenter AsiaOil has been the impetus for a couple of these posts and this one is no different. Here’s the comment that triggered this:

I bet you thought that the shootout string could not last at 5-1 right? Not every situation ends in a 50-50 split in the short to medium term. Have you ever considered that the Oilers are just simply good at the shootout? Hemmer, Gagner, Horcoff and Pisani are solid and Garon has been a wall this year. The percentages certainly favor the Oilers - but by how much? How much is luck? How much is skill? You don’t know and neither do I - that’s my point.

I’m bear baiting because the numbers boys are often just a bit too smug about their powers which are clearly quite limited given the basic statistical tools applied and the nature of the game. A lot of the individual analysis you guys do is great and very interesting - but far too simplistic when applied to complex, dynamic situations such as the future performance of an NHL hockey team over a season.

I’ll start by acknowledging that I think that the complexity of forecasting NHL performance when looking at a reasonably sized chunk of the immediate future is entirely overstated by AsiaOil - if you’re really paying attention and looking at the right areas, I suspect that you can get most of the things in the ballpark, most of the time. There’s always going to be randomness but it doesn’t overwhelm the skill, which can be discerned through the results that guys put up. Of course, certain things - injury and goaltender performance in particular - are a lot harder to predict. Those two factors can swing a team pretty wildly in one direction or another.


The reason I find AO’s bear baiting so baffling (and the reason I’ve written two separate posts on it within a week) is that I think that he’s picked the wrong time to lecture the numbers folk on how their analysis is “far too simplistic when applied to complex, dynamic situations such as the future performance of an NHL hockey team over a season”. Maybe I’m wrong and he’s right, I just don’t think that the Oilers particularly support his claim.

Anyone can surf over to behindthenet.ca and see the Oilers even-strength performance for themselves: 27th in ESSF/60, 21st in ESSA/60, 14th in ESGF/60, 7th in ES shooting percentage, 26th in ES save percentage and 27th in ESGD/60. But for an amazing year shooting percentage wise, they’d be absolutely horrible at ES. As it is, they stink. I don’t see how this makes his case.

On to the PP (I’m using 5v4 as a proxy here because that’s how Gabe has his site set up but the Oilers are 30th overall, so it’s not like the 4v3 and 5v3 numbers are going to bail out this bag of stink): let’s see, 28th in PPGF/60, 30th in PPSF/60 and 21st in PP shooting percentage. I get that Sheldon Souray and Joni Pitkanen being out has hurt but neither of these guys has done much on the PP when they’ve played and it’s not like the Oilers PP has been any better with those guys in the lineup either.

The PK has probably been the highlight of the Oilers season so far (again, 4v5 numbers here) and it’s thoroughly middle of the road. I suspect that a lot of the numbers guys would have said “Yeah, sounds about right” if told in August that the Oilers would be averagish on the PK and terrible on the PP and at ES.

The only reason that I can see for such bear baiting and general taunting of the numbers guys for being wrong lies in the shootout results, although he’s also written that there are “…just too many intangibles like injuries, chemistry and luck” to predict performance - incidentally, I think that the numbers guys tend to be the biggest boosters of luck out there - I can’t see how those would really bear on the shootout results and I think AO acknowledged that it was the shootout performance behind the Oilers stunning run to a three way tie for ninth with two teams that have games in hand on them in the post I’m citing.  In fairness to the numbers guys, the position that AO was advancing was that the PP would be very good and that they’d be pretty decent at ES.  The numbers guy consensus, to the extent that there was one, was a hell of a lot more on the mark.  In fairness to AO, if I understood him correctly, he wasn’t saying that his view was any more valid than the one advanced by the pessimists, only that there was too much faith being placed in “…a bit of dodgy math on the back of napkin and a truckload of assumptions” and that he saw things that were positives.  I’m sure that he’ll correct me if I’ve misrepresented his position.
As I assume that most people reading this are aware, the Oilers have taken part in 11 shootouts this year, winning 10 of them. I’m not so sure that the Oilers winning percentage in the shootout is the really amazing thing - as I’ll set out below, the number of shootouts in which they’ve played is the really phenomenal part. Their success here is really two pronged: first, the record and second, the number of games. Both bear some examination.

Assuming, for the sake of argument, that the Oilers are a 50% true talent team in the shootout, a 10-1 streak could be expected to occur about 5 times in a 1000 11 game streaks. The odds of a five game winning streak for such a team would be about 3 in 100. To answer the first question raised by AsiaOil’s comment, if I had turned my mind to the question of whether the Oilers 5-1 start in the shootout was likely to continue, I would have thought it unlikely unless I believed that the Oilers were about an .870 winning percentage team on true talent in the shootout, which I lean towards thinking isn’t true.

This raises the question: what would an .870 true talent winning percentage team look like in terms of their shooting percentage and save percentage? I’ve simplified things to take a look at that question; what I’ve done is run trials based on three shooters per team and a given team shooting percentage and save percentage. I suspect that this simplifies things too much but it illustrates my general point and seems to mesh relatively well with the results of the past five years. The x axis here is save percentage; the y is shooting percentage. If you find the most applicable set of save percentage and shooting percentage, it appears to be a good approximation of the winning - most teams seem to fall within +/- 1 win of their expected shooting percentage according to the chart. The orange area is the area in which a team’s shootout results would need to fall in order to expect the Oilers current winning percentage on the shootout.


From that, I’d take the position that, in and of itself, the Oilers shootout performance is pretty amazing. I won’t say that they’re outperforming this chart, because it doesn’t appear that they are. I’ve got some doubts as to whether this performance is indicative of the actual skill of the various Oilers who are involved in the shootout - the chart below sets out their results prior to this year and this year’s results - but I’m willing to give them the benefit of the doubt and assume that their results in the shootout this year were earned on the basis of skill. I don’t know that I believe that - I suspect that they’ve been awfully lucky (Hemsky’s goal against the Wild and Pisani’s winner against Vancouver are stuck in my mind) but I don’t know that there’s enough here to say that this is really luck for the Oil.

That, however, isn’t the end of the questions that arise out of Edmonton’s love affair with the shootout. The winning percentage by itself isn’t enough to have the Oilers in a three way tie for ninth - there’s been a simply incredible volume of shootouts. The sheer volume of games that they’ve played that have been tied through regulation and overtime is necessary to cover for their craptacular performance in games that don’t make it to the shootout - 6-15-1. We’re in a bit of a better position here because we can compare the Oilers performance against NHL history - there’s so little information on the shootout that it’s hard to tell whether those results are really indicative of skill or just chance.

As of the Vancouver game, 11 of the Oilers’ 34 games have been tied after 65 minutes, a rate of 32.3% of games. Over the course of a full season, this would be the third highest rate in NHL history. The Oilers rate is all the more amazing because of the relatively recent introduction of 4 on 4 overtime. The highest rate posted by a team since the NHL went that route is that of the 2003-04 Minnesota Wild, who ran up 20 ties - Edmonton is on pace for 26 games tied after overtime. If I can anticipate an argument here, I don’t think that the evidence would support the argument that there’s some skill in taking a game to a shootout - after correcting for the incidence of ties (which tend to increase as goals per game fall and parity increases), the correlation year over year in tie games by team is 0.02. The general theory amongst the people who write about sports through a lens of numbers is that if a result isn’t repeatable, it’s probably not due to a skill - that would suggest that while the Oilers performance in the shootout may or may not be due to their skill, the frequent opportunities to use those skills has its genesis in luck.

Really, I would argue that what we’ve seen to this point in the season is a confluence of two things: a remarkable rate of success in shootouts, which could be due to luck or skill, and an historically remarkable rate of games tied at the end of overtime. I don’t speak for any of the other statsarazzi but if asked at the start of year, I would have said that such a thing was possible but extremely unlikely. As an Oilers fan, I don’t take much solace from what’s happened so far this year because I believe it extremely unlikely to continue.
I’m of the view that all of the doom and gloom predictions for the season look justified to this point in time because even with the shootout performances and the frequency with which those performances have occurred, the Oilers are a team that’s tied for 9th in the conference. One of the teams with which they’re tied has four games in hand, the other has two. The Blackhawks sit two points back in 12th place with two games in hand, the Predators sit four points back in 13th place with four games in hand and the Coyotes are tied with the Predators while having three games in hand on the Oilers. If not for Edmonton’s current edge in games played over a lot of teams, this likely wouldn’t even be a topic for discussion because the rose tinted glasses set wouldn’t be drawing courage from standings that show the Oilers on the fringes of contention.

So really, in conclusion, if AO thinks that the Oilers near historic rate of games that have made it to the shootout and Oiler results in the shootout proves the guys who said that this team was nowhere near good enough in the summer wrong, well, it’s a free country. Even if it continues, they’re probably not a playoff team and the numbers would suggest that they’re unlikely to continue playing this many shootouts. I guess I’ve got nothing to really say if people want to draw solace from the fact that the Oilers have done so well in the shootout but I’d argue that those results don’t tell us all that much about the Oilers as a hockey team and that all of the numbers that do tell us something suggest that this team is a hell of a long way from a playoff appearance, let alone contention for the Stanley Cup. The application of simplistic analysis to complex, fluid dynamic situations that I engage in from time to time here may be utterly without worth but the 2007-08 Oilers aren’t the case on which to hang that conclusion. But for an astounding number of shootouts, something that looks pretty random to me, they look for all the world like a flat out bad hockey team.

Just like I said in August.