• Sifting Through The Ashes I: Contrasting the Oilers and Sharks at 5v4, 2008-12

    by Tyler Dellow • November 16, 2012 • Hockey • 5 Comments

    One of the many things about Brian Burke that I find irritating is his consistent rejection of the use of data and analytics in hockey. On the one hand, I understand it: guys like Burke, who are on the inside, have a powerful interest in preventing the view that outsiders can have something useful to say about hockey from taking root. On the other hand, I’m quite comfortable saying, on the basis of the stuff that I’ve seen over the past few years, that data and analytics, even if just practiced on the limited information that the NHL bothers to gather, can provide good insights into hockey.

    One of the places where Burke does his schtick is at the Sloan Conference every year. I’d tossed in a proposal this year, kind of motivated by Burke being adamant that there was nothing data could tell us. Sadly for me, it appears that that’s gone nowhere, so I’m going to roll the data and information out here over the next couple of weeks. Beats talking about the lockout. I recognize I’m talking to a friendly audience, but this is as much of a proof of concept as anything else: could gathering additional data provide us (or an NHL hockey team spending $50MM+ on hockey players) with valuable additional information?

    One of the things that’s baffled me for a long time is why the Oilers power play is so inept. Specifically, I wonder why they’ve basically been a horror show at generating shots for the entirety of the last CBA. The data that the NHL currently publishes does a good job of identifying which PPs are good and which are terrible but it does a poor job of telling us why or otherwise describing the differences between them.

    I’ve talked before about what I kind of perceive as being the mystery of Ales Hemsky: he’s a guy who’s scored a lot of PP points but the Oilers haven’t been particularly good at scoring PP goals. If we can agree, in the abstract, that points are only relevant insofar as they indicate a contribution to scoring goals, you’re left with a question about whether or not Hemsky’s any good on the PP. If the way in which he scores a lot of points limits the capacity of the Oilers to score goals, is he really good on the PP?

    There’s a lot of smoke around the Oilers problems on the PP but there isn’t a lot of light. The guys in the 300s have their answer: SHOOOOOOOOT! I’ve muttered about really slow movement of the puck and theorized that the Oilers PP doesn’t have enough puck and player movement to generate shots. Some might say that they just have bad players on the PP. Nobody, as far as I know, has engaged in any sort of an objective investigation of the issue, trying to contrast what the Oilers do with what a team that generates a lot of PP shots does.

    “Hey,” I thought “this is something that I could do!” I figured that, as a starting point, I could take a kick at figuring out what’s wrong with the Oilers 5v4 PP (I’m just going to say PP from here on out but I’m dealing exclusively with 5v4) by contrasting it with an awesome PP. Enter the San Jose Sharks. The table below summarizes the data for 5v4 between 2008-2012 – in other words, the period when Todd McClellan was in charge of the Sharks. This is an important point in time because the Sharks were pretty middling at generating shots in 2007-08 at 5v4, despite having some awfully talented players. McLellan moved over from Detroit too, a team that’s known for generating a lot of 5v4 shots.

    The table is sorted by PP shots/60. You see San Jose at the top and Edmonton at the bottom. The columns are pretty straightforward, although the last one on the right warrants some explanation – it’s simply a quantification of how many 5v4 goals a team has scored in each season, relative to the NHL average. So, for each of the last four years, San Jose’s been about 15 goals better than average; the Oilers about two worse than average.

    I’ve tweeted a bit about this off and on over the past few months and when I mentioned I was interested in why the Oilers PP stunk, I was met with a response of “Stop living in the past – the Oilers had an awesome PP in 2011-12.” The Oilers were third in the league in GF/60 on the PP in 2011-12 which is great…except that it was almost all shooting percentage. They finished second in the NHL in shooting percentage and 25th in SF/60. Of the twelve teams who’ve been better than the NHL average of 6.2 GF/60 at 5v4 over the last four years, nine of them were also in the top twelve in terms of S/60. The worst number in terms of SF/60 amongst those teams is 48.7 S/60, which is still considerably ahead of Edmonton’s number last year. In short, the Oilers’ inability to generate shots on the PP is still a problem, no matter how much they scored at 5v4 last year.

    I re-watched all of the Oilers/Sharks PPs between 2008-09 and 2011-12 in an effort to generate data that quantifies the differences between the two teams. The table above summarizes the 5v4 data for the two teams in those 16 games. As you’d expect, San Jose is awesome at generating shots – they’ve generated 5v4 shots against the Oilers like some teams generate 5v3 shots and the Oilers have been pathetic – they’ve generated shots like some teams generate 5v5 shots.

    (Aside: after doing this, I happened across a post at the Footy Blog with a cool quote from Juergen Klopp, manager of Borussia Dortmund, who might be in the top three teams in the world at the moment. Klopp said:

    For me the best analysis is to watch the game again. I know it’s very old fashioned. Tape in, forward and rewind, forward and rewind…a thousand times…spent 5 or 6 hours on a 90 minute game. I haven’t been able to do it any faster. But to be clear: this was my education, no book or seminars or anything from renowned trainers. 10 games a week and I usually started before breakfast.

    The post is about how technology could democratize soccer management and is written by Richard Whittall. Whittall comes highly recommended – according to Ben Massey, he’s one of only two writers in the history of the world who is better than Ben Massey. I digress.)

    I’m going to start with simply the amount of time that the puck spent in the offensive zone during the PP. It’s startlingly similar. When the Sharks were on the PP, they spent 47.3 minutes, or 61.1% of their total PP time, in the Oilers zone. The Oilers spent 38.97 minutes, or 54.4% of their PP time in the Sharks zone. At the risk of stating the obvious, the marginal difference in PP zone time doesn’t account for the massive difference in shooting rates. If you generate a shooting rate using just offensive zone time as the denominator (which is sort of sensible, given that there aren’t really any shots taken from outside the offensive zone with any realistic chance of going in), you come up with 120.5 S/60 for the Sharks and 61.6 S/60 for the Oilers.

    The difference in zone time, slight as it may be, warrants some further examination. The Sharks spent 30.12 minutes outside the offensive zone while on the PP against the Oilers over the past four years and the Oilers spent 32.72 minutes outside the offensive zone against the Sharks – pretty similar. The difference in attempted zone entries is shocking though. The Sharks attempted to enter the Oilers’ zone 159 times in that time; the Oilers attempted to enter the Sharks’ end of the ice just 122 times. Put another way, the Sharks attempted to enter the Oilers end of the ice every 11.4 seconds when they were on the PP and not already there. The Oilers averaged 16.1 seconds of time outside the offensive zone per attempted zone entry.

    First, I want to look at what happened during those zone entries. I broke the zone entries into two types – carries and dump-ins. Passes over the blue line were treated as carries. Interestingly, to me anyway, there wasn’t really a huge difference between the two teams in terms of the mix of carries and dump-ins. The Sharks attempted to carry the puck in 71.1% of the time; the Oilers attempted to carry it in 73% of the time. Basically the same.

    Let’s look at the success rates on those attempted zone entries. The success rate on attempted carries is basically the same – if the Sharks had successfully entered the zone by way of carrying the puck three more times or the Oilers three fewer, they’d be tied. There’s not much to pick between the two teams here. The differing success rate on the dump-ins seems a bit more significant. This is just an observation but the Sharks seem to run dump-ins as a more deliberate play. They seem to happen more when Ryan Clowe is on the ice and they seem to involve firing the puck hard around the boards for Clowe to recover. Oilers’ dump-ins seem to be more ad hoc, like a planned carry has broken down and you end up with some guy backhanding it in with nobody going hard for the puck or Souray just kind of aimlessly bombing it in from centre ice.

    Similarly, the Sharks’ carries into the zone just look cleaner, something I talked about here – they pretty clearly look to isolate a defending player in a 2 on 1 at the blue line and it works a lot. I wrote in the summer about one of their zone entry plays – they’ve got a few of them, but they all seem to be intended to build a framework from which 2-on-1′s are created at the blue line, if the entry is even contested.

    On this data then, you can attribute some of the Oilers’ lack of zone time against the Sharks to an inability to gain the offensive zone by dumping the puck in and recovering it. Some of that is made up for by their superiority in gaining the line when they attempt to carry the puck in. Really, once an attempt to gain the offensive zone is made, it’s close to a wash: San Jose succeeded 69.1% of the time against Edmonton; Edmonton succeeded 68.9% of the time against San Jose. The more critical difference here seems to be in the amount of time that’s needed to make an attempt – Edmonton took 41% longer to put an attempt at the Sharks’ blue line together.

    I’m going to leave this here for now but want to make a couple of final points. First, an obvious area to go back and look at is why the Oilers take, on average, so much more time than San Jose to attempt an entry of the offensive zone at 5v4. Is this an example of the utility of a puck moving defenceman? San Jose, of course, acquired Dan Boyle in 2008, while the Oilers have run a number of different defencemen as their PP guy since then: Tom Gilbert, Corey Potter, Sheldon Souray, Barbaro, Jeff Petry – it’s been a bit of a revolving door, with neither Souray nor Potter being particularly impressive as puck moving guys. That might be a question I try to address as I move forward with this. My gut feeling is that this is going to prove not to be much of a factor – the Sharks have taken a lot of shots when Boyle hasn’t been on the ice – and that it’s more to do with the Oilers lacking a good structure when it comes to exiting their own end of the ice and entering the Sharks’ zone but we’ll see.

    Second, it should be obvious that this is potentially very useful information, if you had it on a league wide basis. If you’re a guy charged with running a successful PP, how can you have any degree of certainty about where you’re good and where you need to improve if you don’t have the thing broken down into a series of digestible parts that you can then compare against other teams? Just knowing who’s good doesn’t help you too much if you’re trying to run an NHL team; you need to know why they’re good. Data like zone entry success rates and seconds per zone entry attempt would provide a useful measuring stick. If it was gathered.

    Next post: A closer look at the causes and implications of the Oilers’ dithering in attempting to gain the offensive zone.

    Email Tyler Dellow at mc79hockey@gmail.com

    About Tyler Dellow

    5 Responses to Sifting Through The Ashes I: Contrasting the Oilers and Sharks at 5v4, 2008-12

    1. Triumph
      November 16, 2012 at

      Very cool, can’t wait to read the conclusion, especially as the Devils just hired the assistant coach who ran the SJ power play.

      I do have one philosophical issue with what you said up top – I’ve been thinking lately that metrics don’t have much use somewhere like the NFL, where A: quantifying more than half the players’ performance is really hard B: since the performance of the people whose numbers are easily counted relies on those whose aren’t, those numbers become suspect, C: so much of the unquantifiable play relies on scheme and strategy than necessarily on that player’s performance and D: it’s really not that hard for NFL personnel to watch all their teams’ plays over the course of a season, know what the players were supposed to be doing, and figure out whether or not they accomplished that goal. Stats might tell you have bad players, but you also might have bad schemes and bad frequencies. Confusing these two things could seriously damage your team.

      I think something like that could be going on here, where you are identifying what’s right or what’s wrong via numbers, but I’m not sure yet how it can be translated down to ice level to help out individual players, and I think one of the larger unsolved issues in the NHL numbers community is identifying which guys excel on the PP and which guys are just along for the ride. (A troll would here point out that Nielson numbers would help, and part of me really would love to put this outside of parentheses just to see how you’d react.) Perhaps that’s coming in Part 2, but short of that, unless NHL people are unaware of general principles that govern the power play (carrying is ceteris paribus better than dumping, zone time is better than not zone time, the zone entry is likely more important than the set up of the power play once the puck is in the zone), I’m not seeing the breakthrough for a non-stats person.

      • Tyler Dellow
        November 16, 2012 at

        I think something like that could be going on here, where you are identifying what’s right or what’s wrong via numbers, but I’m not sure yet how it can be translated down to ice level to help out individual players, and I think one of the larger unsolved issues in the NHL numbers community is identifying which guys excel on the PP and which guys are just along for the ride.

        I think the first step is identifying what’s right/wrong via numbers and then considering whether you can go further with individual players. I’ll be honest: I think that some of this is systemic. The Sharks got A LOT better when Todd McLellan showed up. The Oilers, no matter who they ice, are horrible. Given that it’s awfully hard to find much connection between players and team level PP stats anyway, I’m convinced that there’s something else to it.

        All of which is to say that I think that the first step is figuring out where the Oilers are different than the Sharks. That’s not the same thing as seeing “These players are bad” but trying to identify the objective differences. Your criticism of it may have merit but not yet, I don’t think – give me time to earn it. ;)

        A troll would here point out that Nielson numbers would help, and part of me really would love to put this outside of parentheses just to see how you’d react.)

        They might. Of course, that doesn’t tell you if someone’s piling up numbers because he plays with a defenceman who moves the puck more urgently. Also, if you collect them and mash ES+PP together, you’ll never know anyway.

        unless NHL people are unaware of general principles that govern the power play (carrying is ceteris paribus better than dumping, zone time is better than not zone time, the zone entry is likely more important than the set up of the power play once the puck is in the zone), I’m not seeing the breakthrough for a non-stats person.

        Well, again, give me a little time. I assume NHL people know this (although you’ll more than occasionally hear guys on TV talking about how you just gotta get the puck deep and get after it, something that this suggests is a bad ploy, as does Eric Tulsky’s work at NHL Numbers). The 16 seconds versus 11 seconds thing strikes me as really interesting though. Assuming that this is representative of the past four years as a whole, which I suspect it is, you wonder if the Oilers are aware of it and what steps they’ve taken to try and address it. I bet that the answer is a) nope, unaware (Renney didn’t even know how bad they were at generating shots relative to the league) and b) you don’t try to fix problems you don’t know exist.

        • Triumph
          November 16, 2012 at

          Right, like I said, I want to wait for Part 2.

          “The Sharks got A LOT better when Todd McLellan showed up.”

          But the Red Wings stayed the same, meanwhile the Oilers stayed bad no matter who they hired. So I’m not ready to pin this on McLellan just yet, especially when you consider the turnover on the PP from 08 to 09 – leaders in ice time total, 100 minutes minimum:

          Thornton
          Marleau
          Rivet
          Michalek
          Cheechoo
          Carle
          Pavelski
          Roenick
          Ehrhoff
          Ozolinsh
          Setoguchi

          I mean, Sandis Ozolinsh ran out the last of his career getting 4:33 per game in PP time, although his shots on wasn’t too bad. Plus they got Brian Campbell that year.

          08-09 the cast radically changes:

          Boyle
          Thornton
          Marleau
          Pavelski
          Vlasic
          Ehrhoff
          Blake
          Clowe
          Setoguchi
          Michalek
          Cheechoo

          They switched out: Ozolinsh, Rivet, Roenick, Carle for Boyle, Clowe, Blake, Pavelski, Ehrhoff. That feels like a large upgrade.

          you wonder if the Oilers are aware of it and what steps they’ve taken to try and address it.

          I very much doubt they are. It is, however, an odd place where numbers and coaching intersect – I’ve no doubt that puck movement/alacrity is considered paramount on the power play. Still, you looked at the top and bottom, the difference of which is 21 shots per 60 minutes. Back of the envelope math suggests teams are on the power play for about 525 minutes a year, and you’ve got the Oilers wasting 10 seconds per minute or so, so let’s assume their shot rate during this time is 0 and that the Sharks are at their normal shot rate. This results in an extra 90 power play minutes and… 11 goals a year. Okay, that is a lot. Consider me converted, although you’re not taking into account the strength of the PK units either – I imagine the disparity there is pretty great too, so one has to expect some of those goals to be shaved off by that alone. Then you have to consider personnel – the Sharks were probably good, the Oilers in general were not – could the Oilers even get to that time even if they wanted to?

          Anyway, a lot of work clearly went into this so don’t think I’m doubting the purpose of this – I’m just saying I don’t think it converts a non-numbers guy.

          • dawgbone
            November 17, 2012 at

            The Red Wings staying the same could be a product of the fact that they had so much success that there was no need to change much. I mean you’ve got Lidstrom, Dasyuk, Zetterberg and Holmstrom as 4 vets who’ve been on the PP for several years now. If they say that this is working, you pretty much let them do what they’ve been doing.

            We’ll see how Detroit adapts as those guys leave (Lidstrom is already gone).

    2. November 16, 2012 at

      Off the top of my head: This is the NHL. Every single team has access to 5-7 players who can competently run a power play. So system / coaching is pretty much everything. We witnessed this in Montreal last season, after Gauthier fired Martin (and fired Pearns, who specifically ran the power play, after 10ish games into the season). Gauthier did some nice thing in player procurement, but his inability to understand the percentages is absolutely stunning. I groused extensively about it on my blog last winter, and I’ll summarize part of my argument here.

      The habs generated +37/-7 scoring chances per 60 minutes of play in 2009, +34/-5 in 2010. Their shot rates these season were 57 and 53 shots / 60, according to Behind the net. Both seasons, they had a 21% chances conversion rate.

      This past season, under Jacques Martin, the team went with a new PP pattern, a very interesting one, built around the strength of the players available. I would describe it this way: two inverted point-man, usually Subban on the left and Plekanec on the right. The 3 forwards were usually massed on the *right* half of the ice, one guy either in front of the net or higher in the slot (Gionta or Cole), a second guy against the wall at the faceoff dot’s level (a shooter, either Cammalleri or Pacioretty) and a 3rd guy under the goal line, in the right corner (a passer, usually Gomez or Desharnais). I won’t go into any further details, but suffice to say that this setup was tailor-made for the players available, each being used to their strengths.

      And it worked, just like the previous Martin/Pearns pp worked: they hovered around +40/-8 chances per 60, but had a nightmarish conversion rate, around 10-12%. So you look at this and understand things will smooth over. Gauthier and many people in Montreal clearly didn’t understood that. Because they allowed a few goals early in the season, a whole bunch of people started wailing about how many chances they were giving up on the PP because they used a forward as a point man and how the PP was doomed unless Markov came back. Martin never budged and kept using his PP setup. Gauthier first fired Pearns, then traded for Khaberle to fix a problem that didn’t exist (the PP) and aggravate a problem that actually plagued the team (a defensive corps that was, beyond Subban and Gorges, flat out bad at ES; when Gill and Diaz are your 2nd pair, you are up shit creek).

      Then Gauthier fired Martin and his replacement, Cunneyworth, trashed the Martin PP pattern and instilled a new, everybody stand still around the offensive zone pattern. The result was a steaming pile of shit: +25/-7 scoring chances per 60, but hey, a 23% conversion rate! I’ll add that many of the habs PP goals under Cunneyworth came when the players reflexively switched back to the old pattern and then proceeded to string a few scoring chances together.

      So there. It’s only my limited experience, but I’m pretty certain when it comes to the PP, coaching is everything. And by coaching I mean “seeing how your best players offensive abilities complement each others and use them accordingly”. Imagine that.

    Leave a Reply

    Your email address will not be published. Required fields are marked *