One of the things that I think would be the most interesting part of any general manager’s job is the struggle to understand where your team is now, in terms of performance and the challenge of getting your team to where you want it to be - presumably, a lengthy spell amongst the NHL elite.

This topic came to my mind a week or so ago, as I had Stauffer’s show on for some background noise while doing some work. They were talking about the Oiler and, if I recall correctly, Stauffer was expressing the opinion that an Oilers playoff miss next year should jeopardize the careers of Kevin Lowe and Craig MacTavish. It might have had something to do with Darryl Katz; I don’t quite recall. Anyway, it was interesting to me because I kind of think that the playoffs are further away than they appear. You can’t really get a solid handle on that though without having some sort of a handle on what really happened this year.

There’s a cool passage in Moneyball where Michael Lewis talks about run differential in baseball with Paul DePodesta, then the assistant GM of the A’s, now living in his mother’s basement.

Before the 2002 season, Paul DePodesta had reduced the coming six months to a math problem. He judged how many wins it would take to make the play-offs: 95. He then calculated how many more runs the Oakland A’s would need to score than they allowed to win 95 games: 135…Then using the A’s players’ past performance as a guide, he made reasoned arguments about how many runs they would actually score and allow. If they didn’t suffer an abnormally large number of injuries, he said, the team would score between 800 and 820 runs and give up between 650 and 670 runs. From that he predicted the team would win between 93 and 97 games and probably wind up in the play-offs. “There aren’t a lot of teams that win ninety-five games and don’t make it to the play-offs,” he said. “If we win ninety-five games and don’t make the play-offs, we’re fine with that.”

The A’s ended up scoring 800 runs and allowing 653. Not bad work by DePo.

I would assume that hockey teams or, at least the ones that are well run, go through similar processes. It’s harder to do this in hockey, for any number of reasons but, unsurprisingly, I do think that there’s a lot that can be gained.

A major distortion is the effect of the OT rules creating three point games in circumstances that aren’t particularly similar to NHL hockey: first four on four, then a shootout. This point should be of particular interest to the Oilers because they had some success in games that got to OT last year. The first question I’m interested in, as part of the broader question is this: how repeatable are results in games that get to OT?

OTSOL3

I did some digging - the results are in the sidebar. What I’ve done is this. We have two seasons post-lockout for which we compare a team’s OT/SO stats with those they compiled in the following season. Obviously, we can’t yet compare the 2007-08 results to anything because we don’t have any 2008-09 results. The first chart on the left looks at points percentage. I’ve separated the 60 teams that I have to look at into six groups of ten, from best to worst, in each of two separate measures.

The first measure is games that went to OT/SO, as compared to the league average. What I did, for each of the 60 team seasons that took place in 2005-07, was divide a team’s OT/SO GP by the league average. For instance, in 2005-06, the average NHL team played 18.73 OT/SO games. The Oilers played 26. 26/18.73 = 1.39. That’s a big number - the 05-06 Oilers are second in the group of 60 in OTSO GP/LG AVG OTSO GP.

If you examine the chart, you’ll see that volume of OT/SO GP in one season appears to be a pretty poor predictor of volume of this games the following season. In year one, the teams in the group that played the most OT/SO games averaged about 24 each; the teams in the bottom pairing averaged abou13. In year + 1, neither of those groups was at either extreme; the spread between the groups that played the most and the fewest games respectively was about 3 GP.

Now look at the points percentage. The regression towards the mean seems ever more aggressive there to me. Assuming that all of those teams played league average games, the teams in the best group picked up about an extra 8 points over the teams in the worst group in the first year. In year + 1, the difference is down to about .6 points.

How does this apply to the 2007-08 Oilers? Well, they were outliers in both OT/SO GP/LG AVG and points percentage. In fact, of the 90 post-lockout team seasons, they rank third and second respectively. If both of those “strengths” of the 07-08 Oilers were in areas that seem to show a strong tendency to revert to average - as the “ability” to play OT/SO games and the “ability” to win them seemingly are - one wonders if, perhaps, last year’s Oilers weren’t quite as strong as the standings suggest.

Just for fun, I created something of a junk stat to try and measure the impact of the NHL’s point system on a team’s points. What it does is measure the difference between the points that a team actually got from OT/SO games and the points that they would have got, assuming (a) that they played a league average amount of OT/SO games, (b) had a .750 points percentage and (c) had a .500 record in any games that I’m now treating as regulation games.

To give an example of how this works, the 07-08 Oilers played 25 OT games in a league where 18.13 was average. They earned 44 points in those games. 18.13*0.75*2 (to convert from percentage to points) is 27.2 points. That leaves 6.867 games in which I’m treating them as having a .500 record for 6.86 points. Add it up and you get 34.067 points from games in which they actually got 44 points. That’s the biggest positive difference any team has posted in this junk stat in any post-lockout season.

The effect of wild variations in OT/SO results actually looks to be pretty small; applying my formula to every season since the lockout, 2/3 of teams are within +/- 3.5 points of 0, which represents playing a league average amount of OT/SO games with league average results.

If I’m Paul DePodesta, I’m looking at my team and thinking that it “really” got 78 points last year; if I want to realistically expect to contend for the playoffs, I think that I need to add about 8 wins to my roster. And that’s before I consider other things, like shooting percentages and injuries.

It’s going to be interesting to see what happens with the Oilers next year. If they put up 88 points again but do it with average OT/SO results, that would, to my mind, represent a great leap forward. Listening to Stauffer though, and reading some of the off-season comments from Oilers management, I get the impression that that might not be the common perception. Subject to my thoughts on shooting percentage and injuries, I think that overdelivery is going to be tougher than the Oilers might hope next year.

For those interested, the results that teams have piled up over the past three years are provided in the chart below. Flames fans must be frustrated.

OTSOL1