• New Metrics 2

    by  • September 19, 2013 • Hockey • 9 Comments

    A few days ago, I wrote about a new metric that I’m playing with, Shot Attempts For/100 shifts with 1+SAF (which I call “SAF/100″), which is an attempt to measure how successful players are at turning one shot attempt into multiple shot attempts. Clever readers may have thought “I’ll bet that you could do the same thing with shot attempts against (SAA)” and they would be right.

    What we’re looking at here is a very specific measure of defensive play: once you’ve given up the first shot attempt of a shift, how good are you at stopping the bleeding? I measure this in the form of a number calculated per 100 shifts. I ask myself this: “Once a team has given up the first SAA with Player X on the ice, how many SAA do they give up per 100 shifts?” If you think it through, you see that 100 is the base score: nobody can allow fewer than 100 SAA on 100 shifts on which they allowed at least one SAA. The closer you are to 100, the better your team team has done at preventing additional SAA with you on the ice one the first one has been allowed.

    As I did with SAF/100, I’m going to show the percentiles first, so that people can get a sense of the range for this. Forwards, then defence.

    Screen Shot 2013-09-19 at 10.55.49 PM

    Screen Shot 2013-09-19 at 10.56.03 PM

    We see some things in there that are very similar to what we noticed when we looked at things at the other end of the ice. Defencemen at each percentile of SAA/100 shifts with 1+SAA (“SAA/100″) tend to be slightly higher than forwards at the same spot once you get away from the best and worst; this, presumably is because there is no such thing as a fourth defence pairing that exists solely to ensure that nothing happens while the real hockey players catch their breath. As with the SAF/100 numbers, the numbers seem have edged up each year, whether because of changes in how the game is played or improvements in how things are counted.

    I’ve put up a list of the guys who’ve put up numbers below 130. As you can see, they tend to be guys who are towards the bottom of the roster, although there’s the odd surprising name in there – Patrik Elias shows up repeatedly, which I find kind of interesting. Even if that’s partially related to the comatose shot attempt counters in NJ, you sort of wonder about what’s going on there that he shows up year after year.

    I like to have benchmarks in my mind when I think about this stuff. For defence in 2012-13, it looks to me like anything under 140 SAA/100 is excellent, 145 is about average and anything above 150 and a guy is really struggling. I’m a little hesitant to try and draw such lines for forwards; I suspect that the benchmarks are about the same as for defencemen but that that’s kind of skewed by the fact that guys who face lines from the other team that are lower in the batting order probably post better numbers by dint of the fact that third and, especially, fourth lines aren’t as good and tend to worry more about what they give up.

    With that said, we should take a look at some individual players. Remember Colin Fraser? Stalwart of the Blackhawks fourth line on their Stanley Cup winning team, traded to Edmonton before the 2010-11 season, had what was widely considered to be a disastrous season in Edmonton and then played the part of damaged goods in the Oilers-Kings deal that brought Ryan Smyth home? Here are his open play Corsi% numbers for 2008-09 through 2013.

    Screen Shot 2013-09-19 at 10.56.25 PM

    So really good numbers for a fourth liner in Chicago, bad numbers in Edmonton and right back to really good numbers for a fourth liner in Los Angeles. “Colin Fraser sucks” may have been a popular expression during his time in Edmonton but it doesn’t really seem like a tenable explanation for his open play Corsi%.

    What if we look at his SAF/100 and his SAA/100? This is where I think things get interesting.

    Screen Shot 2013-09-19 at 10.56.50 PM

    As you can see from this, the big change in Fraser’s Corsi% in Edmonton can’t really be tied to an inability to get/prevent shot attempts after that first one. In 2008-09 and 2012-13, his SAA/100 exceeded his SAF/100 and he still posted a positive Corsi%. In Edmonton, his SAF/100 exceeded his SAA/100 and he had a negative Corsi%. If you translate the numbers into hockey in your brain, it seems to me that an explanation for his lousy year in Edmonton founded on an inability to end things in his own end or to generate multiple shot attempts in the opposition end is wrong. The data doesn’t support it.

    I’m not going to go into this at length in this post but here’s another graph, this time of his ratios of shifts with 1+ SAF to shifts with 1+ SAA.

    Screen Shot 2013-09-19 at 10.57.15 PM

    Hmm. That looks a lot like his chart of Corsi%, doesn’t it? Four years above 50%, with a massive year in 2009-10 and then the Edmonton year. I’m going to explore this at more length in my next post but it’s as if Open Play Corsi% is really closely tied to your ability to create shifts with 1+ SAF and prevent shifts with 1+ SAA (and the ability/deficiencies of your teammates in the same area).

    That isn’t to say that SAF/100 and SAA/100 isn’t important or that it isn’t measuring something, just that we should be aware that there’s possibly something bigger at play. Twitter’s Corey Sznajder had a great quote from Dave Tippett in an article that he wrote on Hockey Prospectus:

    We had a player that was supposed to be a great, shutdown defenseman. He was supposedly the be-all, end-all of defensemen. But when you did a 10-game analysis of him, you found out he was defending all the time because he can’t move the puck. Then we had another guy, who supposedly couldn’t defend a lick. Well, he was defending only 20 percent of the time because he’s making good plays out of our end. He may not be the strongest defender, but he’s only doing it 20 percent of the time. So the equation works out better the other way. I ended up trading the other defenseman.”

    I’ve talked about digging beyond Corsi% and that’s really what this; it’s what Tippett was talking about, even if he processes the information differently than we do. It’s figuring out what matters and how one should weight it in their mind. I’m going to leave this idea here and come back to it in a future post but that’s really what this is all about: improving the information that we have and then processing it better.

    What about Patrick O’Sullivan, another Tambellini acquisition who came to Edmonton and, after a year of not being good, was dismissed with a “Well, he must suck”? Let’s do the same graphs that we did with Colin Fraser.

    Screen Shot 2013-09-19 at 10.57.35 PM

    O’Sullivan infamously came into Edmonton from a season in Los Angeles in which he was having a huge Corsi%. He actually put up a respectable Corsi% for the rest of the year in Edmonton, before things went wrong. Some out of their depth zealots, who like to pretend that serious data people use Corsi% in a way that they don’t, point to O’Sullivan from time to time as a data point in support of the conclusion that Corsi% is a waste of time. “Look,” they say, “if Corsi% is useful, why does Patrick O’Sullivan do well on it in Los Angeles and poorly elsewhere? It must be worthless. Use scoring chances, which are far better.” The “scoring chances are better” position is never substantiated, because it can’t be, because there’s no league wide data.

    We know that scoring chances tightly follow Corsi% and I suspect that if we had the data, we’d find that Patrick O’Sullivan’s scoring chance numbers were great in 2007-08 in L.A. and then he came to Edmonton and things were horrible in 2009-10. That seems to be how scoring chances work. So, in addition to the subjectivity problem, I think scoring chances have the same problem as a metric that Corsi% does.

    Digression aside, let’s look at his SAF/100 and SAA/100.

    Screen Shot 2013-09-19 at 10.58.05 PM

    You can see that things flipped when he came to Edmonton. Again, it’s interesting to me that, despite the fact that O’Sullivan’s SAA/100 was higher than his SAF/100 in the chunk of the season left after the trade in 2008-09, that his Corsi% stayed north of 50%. What if we look at the 1+ SAF/1+ SAA ratio?

    Screen Shot 2013-09-19 at 11.01.34 PM

    Hmm. Again, we see that when O’Sullivan managed to have more shifts where his line had 1+ SAF than shifts with 1+ SAA, the Corsi% was positive. This may seem overly simplistic to people but I think that there’s an important truth here – this is something that I’ll get into in further detail in my next post on the topic. It seems possible to me that the ability to generate or prevent multiple SAF/SAA once the first one has happened is of less importance than the ability to create or prevent that first SAF/SAA.

    What flows from that, logically? Well, in looking to dig below Corsi%, perhaps we should be trying to isolate the things that lead to SAF and SAA and put the majority of our focus there. It would be interesting if, for example, we had a complete set of zone entry/exit numbers for the 08-09 Los Angeles Kings and Edmonton Oilers and the 2009-10 Edmonton Oilers. Is it possible that O’Sullivan was doing the same things, in terms of things that generate SAF/prevent SAA and that what happened was a sudden and horrible change in context, once he left the safe embrace of Anze Kopitar? That this was exacerbated by the replacement of a thoughtful coach in Craig MacTavish with a guy who probably called Sam Gagner “Dave” a few times during the season? It seems plausible to me. Again, I don’t want to go too far down this road now; we’ll explore it a little further in the next post.

    To be clear, I’m not saying that SAA/100 doesn’t matter, only that there’s some reason to think that it may really be a secondary thing compared to the ratio of shifts with 1+ SAF/1+SAA. This is, in theory, an Oilers related stats blog, so I’m going to wrap this post up putting up some graphs of Oiler numbers in this facet of play during the period in which this data’s available.

    Remember how the Oilers went from having Craig MacTavish as their head coach in 2008-09 to having Pat Quinn as their head coach in 2009-10? There’s probably a reason that Quinn didn’t say “We’ll play tighter defence” in this fantastic video that Young Willis unearthed. Check out the SAA/100 numbers for the defencemen (min. 200 Corsi events) who were on the team in both 2008-09 and 2009-10.

    Screen Shot 2013-09-19 at 11.53.08 PM

    Good lord. It’s like the entire team exploded. Every single player performed substantially worse than they had in the previous season. Every. Single. Defenceman. I’ve been amending my thinking on coaching after the past six months or so; I’m coming around to Giovanni Trappatoni’s line on coaches (or managers, as they’re called in soccer) lately – “A good manager makes a team 10 percent better, and a bad manager makes it 30 percent worse” – and what happened in Edmonton in 2009-10 doesn’t really dissuade me from that.

    A final graph, in honour of Ryan Nugent-Hopkins, who signed a contract today that ensures he’ll be in Edmonton for seven more years after the coming season. I joked when the Selke Trophy nominees came out this year that it was the hockey hipster Hart Trophy, in that Pavel Datsyuk, Patrice Bergeron and Jonathan Toews were all beloved by hockey stats guys and maybe don’t get enough credit amongst fans who watch the games instead of their spreadsheets. Plus, all three look like they’d grow terrible moustaches. Here’s what those guys have done in SAA/100 over the past six years:

    Screen Shot 2013-09-19 at 11.53.22 PM

    If you go back to those benchmarks I talked about earlier, you see that, year after year, these guys are amongst the NHL’s elite in terms of SAA/100. It makes a certain amount of sense: they have good defensive reputations. The data would seem to back up the conclusion that, at least at this aspect of defensive play, they achieve elite results when they’re on the ice.

    There’s more to winning a Selke Trophy than this (Bergeron, Toews and Datsyuk all kill penalties and win faceoffs) but RNH’s 136.8 SAA/100 last season is an elite number, right up there with the Selke Trophy candidates. It’s also the seventh best by an Oiler F (min. 200 Corsi events) since this data becomes available in 2007-08. There’s no question of him having been sheltered last year – he wasn’t. (Of note for Eberle fans: Eberle actually posted a number that was slightly better than RNH’s last year at 135.4). RNH has had a good defensive reputation amongst scouts and Oiler fans, I think – it’s exciting to see some context for just how impressive his season was last year in terms of limiting second and third shots. It’s one year but every great defensive centre’s track record starts with one impressive year. Hopefully that was the first of many.

    I just want to underline something as I wrap this post up: to me, these numbers (or any numbers) aren’t really a be-all and end-all of the discussion. They’re a record of performance when a player was on the ice. That being said, what I think they’re good for is providing us with objective data to with which to challenge our beliefs. If someone says (and Lord knows they have) that Tom Gilbert was no good in his own end because he was soft, you could quite reasonably point to his SAA/100 in his last two seasons as an Oiler of 141.8 and 138.1 and say “Look, the Oilers do pretty well in terms of not conceding multiple shot attempts when he’s on the ice.”

    Maybe there’s a reason for that, something that made things easy for Tom Gilbert. Maybe there isn’t. The point is, it’s objective information describing the results that occur when a player is out there. It provides a useful check, in that if it conflicts with your opinion, you might re-examine how you arrived at your conclusion.

    Email Tyler Dellow at tyler@mc79hockey.com


    9 Responses to New Metrics 2

    1. September 20, 2013 at

      Isn’t your interest in shifts with more than one SF/SA a variant of the first-goal fallacy?

      (By which I mean the old chestnut about the first goal mattering a lot, without the narrator realizing they’re effectively splitting their sample into a set of games where the Heroes scored at least one goal, and a set of games where the Villains scored at last one goal).

      So I’m suggesting that, similarly, if I understand what you’re doing correctly, you’re discovering that shifts with at least two shots credited to one side are predictive of succeess, but to do so forgets that your samples split a lot like first-goal games.

    2. Woodguy
      September 20, 2013 at

      If you are going to use SAA/100 as a way of evaluating defensive ability, would it make sense to figure out what a player’s Expected SAA/100 is and then compare it to the result?

      I think this is a way to remove the bias in the data that shows 4th liners as good defensively because they rarely are out against players who are elite of generating shot attempts.


      Player A and B both have 140 SAA/100

      Player A’s expect SAA/100 is 130 (Mike Brown) and Player B’s is 150 (RNH)

      The 140 result, which is the same until context is introduced, now looks much different.

      Since you have created the SAF/100 and SAA/100 data, and the TOI vs. players data is available, you should be able to generate and Expected SAA for every player, given who they have played.

      It might not create recognizable divisions (I expect many players to have similar Expected SAA/100), but it might help interpret the results more and perhaps start down a road to a better Quality of Competition metric.

      Fun stuff Tyler. Thanks for sharing the fruits of all your squinting at your computer screen.

    3. Pingback: Spectors Hockey | NHL Blog Beat – September 20, 2013.

    4. Lloyd B.
      September 20, 2013 at

      I think what you are discovering statistically is what most observers by eye have known for years. If you are on the ice for more shots against then you will have more goals scored against you when you are on the ice. The shifts that become a shooting gallery either way often result in more goals being scored as players are pulled out of position, the goalie is sprawled on the ice, etc. In the one shot and out shifts, you are scoring less goals which you very clearly identified in the excellent work on Gagner and Hemsky.

    5. Pingback: Daily Dashboard September 21st, 2013 - Sabre Noise - A Buffalo Sabres Fan Site - News, Blogs, Opinion and More

    6. Pingback: What is 'open play' hockey? - HockeyAnalysis.com

    7. Pingback: [Bits] Islanders-Devils Preseason Reactions; NHL Suspensions | All New York Islanders

    8. Pingback: [Bits] Islanders-Devils Preseason Reactions; Clutterbuck 4-6 Weeks; NHL Suspensions | All New York Islanders

    9. May 6, 2014 at

      What run away from NS talking you? When NS was first incotdured in 1967, he was already 27 years old – way past the age to be drafted.In spite of that, while other 27 year old doctors at that time pursued their careers, only he and 11 another doctors had to do NS!That’s where he objected in principle.

    Leave a Reply

    Your email address will not be published. Required fields are marked *