An interesting topic popped up on Twitter this morning during the tennis, one that I haven’t written on for a while: overtime and shootout games and what sort of a conceptual framework we should have for understanding them.
There are two schools of thought on this. The first one, which is incorrect, talks about team getting loser points for a loss in overtime or a shootout. It’s understandable that people think this because their frame of reference is the NHL’s previous system. Prior to 1999, if you lost a game, whenever you lost it, you were awarded zero points and may God have mercy on your soul. Now, if you lose a game in overtime or in a shootout, you get a point anyway, so it’s a loser point, right?
Wrong. The second understanding of this, which is conceptually accurate, is one which pays attention to what is happening and when the points are awarded. Prior to 1999, if a game went to overtime, a team had yet to secure any points in the standings. What the NHL really did was change the point in the game at which points are awarded. Now, if regulation play finishes with the two teams tied in the standings, each team is awarded a point. You then play a five minute 4v4 game for an extra point and, if that doesn’t produce a winner, a shootout ensues.
The misunderstanding from the first school of thought is exacerbated by the way in which the NHL presents the standings. Rather than a four column set of standings that accurately portrays what is going on (W-L-T-OT/SOW), they like the simplicity of the three column setup (W-L-OTL). This encourages people to think of OT/SOL points as being the “extra” point when the extra point is actually in the W column in disguise.
This leads to people pointing to teams like the 2011-12 Florida Panthers, who went 7-18 in OT/SO and saying “Pfft. They’re only in the playoffs because of the Bettman point.” Really though, if you ignore everything that happened after sixty minutes, the Panthers collected 87 points, good for fourth in the Eastern Conference. The Panthers weren’t a very good team and fluked into a playoff appearance but it wasn’t because of what happened after sixty minutes. They didn’t do very well after regulation but, fortunately for them, they put away enough points that it didn’t matter.
What about winning that extra point though? Is it a skill that reveals itself over the course of an NHL season? I tend to to think not. If something’s a skill, I expect that it will persist, from one season to the next. Wayne Gretzky was good at running up points one year, he was good at it the next year. It’s a skill. Is getting extra points a skill that’s persistent from season to the next? To test this, I calculated the difference between the average standings points an NHL team piled up after regulation and each team. So, in 2011-12, the average team earned ten extra points in OT and the Oilers earned seven – they were -3.
I then paired each team with how they did the following season: 2005-06 with 2006-07, etc. I did this through to 2010-11 and 2011-12. We didn’t get a full season last year, so I didn’t want to dump that data into it. This gave me 180 pairs of numbers. I then sorted them into ten buckets. The first bucket had the 18 teams with the fewest OT/SOW points relative to league average. I repeated this until the tenth bucket had the 18 teams with the most OT/SOW points relative to league average. I then looked at how the 18 teams in each bucket performed in the following season.
The 18 worst teams in my 180 teams averaged 5.4 SO/OT wins below league average in the first year. The next year, they averaged 0.2 wins better than average. The 18 best teams in my sample averaged 6.1 SO/OT wins above league average in the first year. The next year, they averaged 0.3 better than league average. You’d be hardpressed, I’d argue, to find any evidence of a persistent skill to this.
As you’d expect, the correlation coefficient from year to the next in OT/SOW relative to league average is abysmal: 0.003. This is kind of crazy – if you had two identical teams and one had a really good year in the OT/SO department and the other a really poor one, you could see an 11 or 12 point difference and there’d be nothing actually between the teams. It’s like coaching/managerial career Russian roulette, in that the guy on the wrong end of chance stands a decent chance of seeing his employment terminated.
Let’s contrast this with regulation points. We think that being a good team is an indicator that you’ll be a good team the following year but we know that winning OT/SO games doesn’t seem to be a repeatable skill. Is being better or worse than league average in regulation a repeatable thing? I repeated the same process outlined above for the same seasons and produced the table set out below.
Obviously, the league average for regulation points in a non-lockout season will be 82. So the average team in my worst bucket here gets about 58 (82-24) regulation points in Year 1 and the average team in my best bucket gets about 105 (82+23) regulation points in Year 1. While they both move towards 82 points in Year 2, they aren’t remotely close to being indistinguishable in Year 2 as is the case with the best and worst teams at generating points relative to league average in OT/SO. In other words, while they may not be as good or bad as they appear (or they’re unable to sustain such suck, for reasons like replacing the guy who put JF Jacques on the first line, or such goodness because it’s a cap league), there is a real, persistent difference between them.
The differences get a lot smaller once you’re away from the extremes though. The difference between a 90 regulation point team and a 73 regulation point team are smaller than they appear and probably have a lot to do with health and whether or not pucks go in. Note that the teams in bucket 8 average 90 regulation points in year 1 and 86 regulation points in year 2 while the teams in bucket 3 average 73 regulation points in year 1 and 80 regulation points in year 2.
Why does all of this matter? Well if you’re running a hockey team or interested in hockey analysis, you have to be aware that the British aphorism is wrong: the table lies all the time. The introduction of OT/SO points means that it lies even more often than it did in the past and the way in which it’s presented as W-L-OTL makes the lies all the more cunning. People who want to understand what really happened have to be able to sort the truth from the lies or, put another way, the signal from the noise. The whole trick is figuring out what’s signal and what’s noise and obsessing about the signal while ignoring the noise.
One last table: a look at the how the Oilers did in terms of standings points relative to the league average in regulation and OT/SO from 2005-06 to present, with the 2013 season converted to a per 82 game number.
It’s a pretty cool little table, I think. You can see that the 2006-07 and 2007-08 teams were actually pretty similar overall, with the difference being that the 2007-08 team had the best season relative to the league average at OT/SO that’s yet occurred and the 2006-07 team getting skunked.
That 2006-07 team is pretty interesting – they got 4 regulation points in their final 20 games which means, if you do the math, that they were actually a little bit better than the 2005-06 team in terms of earning regulation points to that point. Unfortunately, the 2005-06 could get OT/SO points and the 2005-06 team couldn’t so they blew up the team and traded Ryan Smyth.
Then you see the 2008-09 team, which actually took a step back towards respectability in terms of regulation. The rebuilding of the team was paying some dividends, even if the team that they were building was unlikely to be one which could compete for the Cup at any point. It was disguised by the big drop in OT/SO points from the previous year though, so MacT left and Steve Tambellini declared that he was in charge and would deal with the few remaining embers and OH MY GOD A 60 FOOT TOWER OF FLAME.
We (in the sense of people who find analytics useful) would probably benefit with a standings page somewhere that presents them in a W-L-T-OT/SOW format. If OTW/SOW points are just noise, standings that had them separated out or presented regulation points would be helpful in terms of a quick look to see if a team is for real or not. Someone, other than me, should get on that.Email Tyler Dellow at firstname.lastname@example.org