• “Moneypuck” *cringes*

    by Tyler Dellow • October 7, 2011 • Hockey • 61 Comments

    James Mirtle’s series of pieces on the inventor of hockey statistics Gabe Desjardins and hockey analytics generally in the Globe and Mail a little while back touched off a bunch of interesting stuff. I always enjoy reading smart people who are intellectually honest and disagree with me. Elliotte Friedman, Tom Benjamin and my buddy Rajeev all fit that description.

    Benjamin wrote a pretty lengthy post at his site that I think warrants some consideration – I encourage people to go read it. With that said, I have significant problems with what he said. Tom quite properly acknowledges that hockey analytics is pretty fantastic when it comes to blowing up media myths. He goes on to say that hockey stats will never provide enough insight though.

    Hockey statistics will never do what we want them to do, which is to effectively evaluate individual hockey players. To give us answers when considering a trade or a personnel decision. To tell us whether this third line winger creates more wins than that number four defenseman. Baseball statistics can produce answers to these kinds of questions, while hockey statistics can only produce more questions.

    Why? Because baseball statistics describe what actually happens in baseball games. The stats add up to runs and to wins. Hockey statistics do not add up to goals. Goals or proxies for goals like shots, shots and attempted shots or even quality shots, underpin all the analyses. None of these statistics say anything about how the scoring chance, the goal was achieved. The actual activities that go into creating the chance are not recorded. The fundamental problem is that hockey has the equivalent of runs, but the hits and walks that create those runs are missing from the statistical package.

    I agree with Tom to a point on this. I can’t agree that, even in their relatively primitive state, hockey stats can’t give us answers when considering a trade or personnel decision. They may not be precise yet, to the point where we can say “Player X is worth precisely $Y.” They can inform our decisions though. We can learn to avoid guys coming off percentage fuelled seasons. (Most of us can; some of us seem to be the sort of people who just keep touching the stove element.) This, in and of itself, is important stuff – a stack of money has been wasted by NHL teams on these guys.

    He’s right though, about baseball statistics describing what happens in a game, while goals and assists do not. Accepting that though, I don’t conclude that hockey stats can never give us more precise answers. I conclude that we need to start generating better statistics.

    Since I first got interested in hockey statistics and analytic stuff back in 2002 or 2003, I have learned a ton of stuff about NHL hockey that I didn’t know before. The impact of randomness relative to the impact of skill on the percentages was news to me. The predictive power of things like Corsi and shots in terms of reflecting skill rather than randomness was too. The limited impact of shot quality was news. The way in which I think about hockey has been completely changed as a result of this stuff. I do, I think, have a more accurate view of things now than I once did. What’s more, I can back up things that I say and think in a way that I couldn’t when I started getting into this stuff. I’ve got a pretty decent track record of pointing out when NHL GMs do completely ridiculous things – I attribute a lot of that to the understanding of the game I’ve gleaned from numbers.

    Tom does, it seems to me, concede the critical question:

    With lesser players – the real challenge in objective evaluation – the individual numbers mean even less because different players contribute in entirely different ways. There are reliable defensive players who seldom make a mistake, but provide almost no offence. There are other defensemen who can make a good pass and score once in a while, but make too many mistakes without the puck. There are forwards who can score but do little to help the team move the puck into scoring position. There are forwards who can win puck battles, kill penalties and even move the puck, but their hands are stone. Goaltenders don’t do anything except guard the net.

    It is the mixture of all the skills, the collective skillset, that produces team strengths and team strengths win hockey games. Team speed. Team toughnesss. Team goaltending. Team defense. Team offense.

    We can’t objectively sort out the individual contributions to those team strengths. Until we can find the hockey equivilent of singles, doubles and triples from the organized chaos of the game, the statistical evaluation of individual hockey players with disparate skills is a mug’s game.

    I think we are, to a degree, closer than he thinks now. I don’t quite agree with him on his hands of stone thing – when you start talking about lesser players in the NHL, you’re talking about players who probably have true talent on-ice S%’s of somewhere between 7% and 8.5% or so at 5v5, with randomness smearing things so much in a single season as to make it effectively a wash for analytical purposes, assuming we’re leaving out the real plugs, guys like Sandy McCarthy.

    There are things we can do to tease out the impact that individual players make on the game. We know how important it is to have an edge in shots over your opposition. What we need to do is to come up with a way to start figuring out how to identify the players who create that and how they create that. The singles and doubles and triples of hockey. Soccer’s enjoying a bit of a tactics moment right now, with people like Jonathan Wilson and Michael Cox writing intelligently about tactics. They (in particular, Cox) are greatly helped by the data that’s available with respect to passes, average position on the field and such things.

    As I’ve become a soccer fan, I’ve become really cognizant of the similarities between hockey and soccer. It’s funny – I think Tom would concede that these exist, because in the comments to his post, he classes hockey and soccer as games that aren’t susceptible to statistical analysis. I’d noticed it before, particularly when watching Olympic hockey, where players hold the puck longer. The dimensions and surface are different, and create some differences between the game, but ultimately, both are fluid games where you want to limit the opposition’s quality shots while maximizing your own. Soccer has, I think, moved a long way in front of hockey in terms of tracking the right stuff to answer these sorts of questions – there was a great article in the Financial Times earlier this year that’s worth reading if you’re interested in quantitative data and sport. An excerpt:

    Yet by the mid-2000s, the numbers men in football were becoming uneasily aware that many of the stats they had been trusting for years were useless. In any industry, people use the data they have. The data companies had initially calculated passes, tackles and kilometres per player, and so the clubs had used these numbers to judge players. However, it was becoming clear that these raw stats – which now get beamed up on TV during big games – mean little. Forde remembers the early hunt for meaning in the data on kilometres. “Can we find a correlation between total distance covered and winning? And the answer was invariably no.”

    Tackles seemed a poor indicator too. There was the awkward issue of the great Italian defender Paolo Maldini. “He made one tackle every two games,” Forde noted ruefully. Maldini positioned himself so well that he didn’t need to tackle. That rather argued against judging defenders on their number of tackles, the way Ferguson had when he sold Stam. Forde said, “I sat in many meetings at Bolton, and I look back now and think ‘Wow, we hammered the team over something that now we think is not relevant.’” Looking back at the early years of data, Fleig concludes: “We should be looking at something far more important.”

    That is starting to happen now. Football’s “quants” are isolating the numbers that matter. “A lot of that is proprietary,” Forde told me. “The club has been very supportive of this particular space, so we want to keep some of it back.” But the quants will discuss certain findings that are becoming common knowledge in soccer. For instance, rather than looking at kilometres covered, clubs now prefer to look at distances run at top speed. “There is a correlation between the number of sprints and winning,” Daniele Tognaccini, AC Milan’s chief athletics coach, told me in 2008.

    It strikes me that there’s probably a lot of stuff that can be mined in hockey, stuff that would tell us a lot about which players are most valuable. As I mentioned above, I’d be fascinated to see what sorts of things correlate well with possession and which players do those sorts of things. We’re getting to the point now that we know a lot of the stuff that is tracked is useless, I think. Hits and giveaways and takeaways…nobody serious cares about the data the NHL is generating there. I have increasing distaste for assists and points.

    The real difference between baseball and hockey is that baseball’s essential data is largely public. The singles and doubles show in the statistical record. With hockey and soccer, that’s not the case – someone needs to figure out what the essential pieces are and, due to the cost and proprietary interest in that not becoming known, it’s unlikely to ever become public record. Soccer data is notoriously limited and controlled by the companies that collect it.

    Hockey’s at least a decade behind soccer though. There’s no reason that has to be the case. There’s also no reason, given the number of insights into the game that have been generated from the relatively crappy set of data that the NHL collects at present to expect that this will continue. You give someone like Gabe or me a database of every touch that happens in a season and where it occurs, and I suspect a lot of really useful stuff would come up. That’s the future. I don’t even think it’s that expensive – if some guy put me and Gabe Desjardins in a room for a year and gave us a million bucks to spend on generating the data, I expect we’d find all sorts of stuff. That’s pretty cheap – if the Oilers did it, we could pretty much pay for ourselves just by stopping them from making one signing. Until that time comes though, when someone goes out and puts that together, there’s still a lot that we can learn from the data that does exist. I’m more of a believer than I’ve ever been that teams that don’t get into this stuff will end up being left behind and having to play a significant amount of catch up.

    About Tyler Dellow

    61 Responses to “Moneypuck” *cringes*

    1. October 7, 2011 at

      I think we are, to a degree, closer than he thinks now. I don’t quite agree with him on his hands of stone thing – when you start talking about lesser players in the NHL, you’re talking about players who probably have true talent on-ice S%’s of somewhere between 7% and 8.5% or so at 5v5, with randomness smearing things so much in a single season as to make it effectively a wash for analytical purposes, assuming we’re leaving out the real plugs, guys like Sandy McCarthy.

      Travis Moen’s on-ice shooting percentage (not his, his teams shooting percentage when he is on the ice) over the past 4 season’s is 5.5%. That means there is about 99.5% certainty that his on-ice shooting percentage is something less than 7%. There is a 95% certainty that it is below 6.5%.

      It is unfortunate that so many people have the misconception that shooting talent and shot quality doesn’t matter. It is not true. The odds that Travis Moen (5.5%) and Sidney Crosby (10.91%) would have more or less the same on-ice shooting percentage over the past 4 seasons if not for luck and randomness is miniscule.

      What I think has happened is people have showed that you can’t discern between any two players shooting percentage to a 95% confidence level over the course of a single season and concluded that it doesn’t exist or matter. This is unfortunate and in my opinion has pushed hockey analytics in the wrong direction (towards counting shots or chances, and away from what matters – goals). Just because something can’t be easily identified at a high confidence level from a small sample size does not mean it does not exist.

      Just my opinion anyway.

      • Tyler Dellow
        October 7, 2011 at

        David -

        The comparison of Crosby and Moen isn’t really apt, since we’re talking about lesser players.

        In any event, I grabbed the data off your site so I could check. Of players who’ve been on the ice for 1000 shots or more over the past four years (481 guys), only 46 can we say with 95% certainty that their true talent on-ice S% over the past four years is not between 7% and 8.5%.

        I’m not denying that there’s a talent here – Tom Awad convinced me. The problem that you have is that most players aren’t Moens or Crosbys. They fall somewhere in between.

        • October 7, 2011 at

          Can you generate a list of the those players? I can probably guess most of them, but it would be nice to note the outliers.

          • Tyler Dellow
            October 7, 2011 at

            Here’s the list of guys who we’re 95% sure aren’t between 7% and 8.5% over the past four years. Note that only ten of them go the wrong way, which is consistent with the point that I made. Of those, three have significant time in Anaheim.

            GABORIK, MARIAN 11.2%
            CROSBY, SIDNEY 10.9%
            RYAN, BOBBY 10.8%
            TANGUAY, ALEX 10.7%
            SEDIN, HENRIK 10.5%
            SEDIN, DANIEL 10.4%
            DOWNIE, STEVE 10.4%
            DUMONT, J.P. 10.4%
            WHITNEY, RYAN 10.4%
            MALKIN, EVGENI 10.3%
            HORTON, NATHAN 10.2%
            GETZLAF, RYAN 10.2%
            HEATLEY, DANY 10.2%
            SPEZZA, JASON 10.2%
            KOVALCHUK, ILYA 10.1%
            PERRY, COREY 10.0%
            KOSTITSYN, SERGEI 10.0%
            RIBEIRO, MIKE 10.0%
            STAMKOS, STEVEN 10.0%
            MORROW, BRENDEN 9.9%
            SEMIN, ALEXANDER 9.9%
            ST._LOUIS, MARTIN 9.9%
            MALONE, RYAN 9.9%
            FINGER, JEFF 9.8%
            STASTNY, PAUL 9.8%
            OVECHKIN, ALEX 9.8%
            THORNTON, JOE 9.8%
            VANEK, THOMAS 9.8%
            BURROWS, ALEX 9.7%
            TOEWS, JONATHAN 9.7%
            SCHULTZ, JEFF 9.7%
            DATSYUK, PAVEL 9.6%
            IGINLA, JAROME 9.6%
            STAFFORD, DREW 9.6%
            ANTROPOV, NIK 9.6%
            BACKSTROM, NICKLAS 9.5%
            MARTINEK, RADEK 6.0%
            WINNIK, DANIEL 5.9%
            PAHLSSON, SAMUEL 5.6%
            MOEN, TRAVIS 5.5%
            WITT, BRENDAN 5.5%
            THORNTON, SHAWN 5.4%
            SJOSTROM, FREDRIK 5.4%
            MARCHANT, TODD 5.3%
            VEILLEUX, STEPHANE 5.0%
            ADAMS, CRAIG 4.8%

            • October 7, 2011 at

              Interesting. We could probably pare down this list further by isolating guys who probably aren’t driving things, but merely shared the ice wither others who were. Shultz, Burrows and Finger jump out.

            • Tyler Dellow
              October 7, 2011 at

              Sure. All the defencemen are suspect too – there are what, four on the list? They aren’t driving things.

            • October 8, 2011 at

              I notice that a large number of the best players by shooting percentage play together on what most people would consider some of the strongest 1st lines in the league. I’d suggest that this might indicate that it takes a lot more “hockey talent” to push shooting percentages by a noticeable amount than to push shot totals. So much so that great players need the help of other great players to get there.

              In a population of 700 players your also going to see some false positives for a 95% confidence test, which might explain the remaining non-elite players without great linemates that ended up there (Antropov, Dumont, S. Kostitsyn for example).

        • Tyler Dellow
          October 7, 2011 at

          Another point – it would be interesting to see what this list looked like with road data only. A couple of these guys – Moen and Pahlsson leap out – played in Anaheim, which has a notoriously generous shot counter.

          • October 7, 2011 at

            You won’t win many arguments that we should use corsi because we can’t trust shooting percentage analysis with a statement talking about the inadequacies in the recording of shots. If what you say is true, it just nullifies all those corsi based analyses as well.

            • Tyler Dellow
              October 7, 2011 at

              Not if the shot counter is equally generous or stingy at both ends and the ratio is what you pay attention to. Which is what it works out to.

            • October 7, 2011 at

              Ratios are fine, but what if I want to find out who the best defensive players are on my team so I know who to play most when protecting a lead or who to match up against the oppositions best offensive players. A ratio doesn’t tell me if the player is a high risk, high reward type player or a defensive player who doesn’t score many goals but doesn’t give up many either. I think there is merit into looking into offense and defense stats separately.

        • October 7, 2011 at

          It may be fair to say that only about ~10% are true outliers, but the outliers are the players we want to be most interested in. It’s the outliers that GMs should try to acquire or avoid.

          Also, as I said before, just because it doesn’t pass a significant test doesn’t mean it doesn’t exist. Id I recall, the work that Tom Awad did showed that shooting talent and shot location accounted for more than 50% of “what makes players good” (see the first post in that series). That seems pretty significant to me and not something that we should ignore.

          • Tyler Dellow
            October 7, 2011 at

            Except that if we’re talking about the guys in the middle of the league, the vast majority of whom are in some statistically indistinguishable blur between 7% and 8.5% – and, while I hate to harp on this, THAT’S WHAT I WAS TALKING ABOUT – then Tom’s point about true talent shooting percentage isn’t going to apply.

          • October 8, 2011 at

            True outliers are certainly interesting, in terms of studying, say, why Sidney Crosby is so good. The outliers on the top end though are generally fairly obvious from the perspective of “hey, this guy is good and you should get him”. I’d argue that makes analysis of them less useful. We don’t need Corsi or whatever to tell us that Sidney Crosby is pretty good. On the bottom side, you do need some analysis to show why Sami Pahlsson, despite his poor S%, is worth signing. And such analysis can illustrate that Moen actually does nothing at all, except drag down S%. So that’s useful, but only on the lower end, or less than the original 10% of players, more like 2%.

            Of course, even if we’re generous and leave the 10% of outliers as being interesting, you’re still only talking about 10% of NHL players, and again, at the top-end outliers, you’re generally talking about the easier decisions. NHL GM’s need information about the other 90% of players, differentiating 2nd line forwards from 3rd line forwards, trying to find out if a 2nd/3rd line guy on a top team can fit the 1st line role for a bad team, or if he’ll always be a true 2nd/3rd line guy, regardless of the team. Trying to find the difference between a 4th and a 7th defenseman. That kind of stuff is where the tough decisions are, and that’s where you need this sort of information, more than anywhere else.

            • October 8, 2011 at

              Of course, even if we’re generous and leave the 10% of outliers as being interesting, you’re still only talking about 10% of NHL players, and again, at the top-end outliers, you’re generally talking about the easier decisions. NHL GM’s need information about the other 90% of players, differentiating 2nd line forwards from 3rd line forwards, trying to find out if a 2nd/3rd line guy on a top team can fit the 1st line role for a bad team, or if he’ll always be a true 2nd/3rd line guy, regardless of the team.

              This is true, but the outliers get much more significant when we consider a 90% confidence test or even an 85% confidence test. If I were a GM I am willing to make decisions, or at least take much more serious consideration, on a 90% or 85% confidence level.

              Besides, the main point I am trying to make is that shooting percentage matters. Even that mass middle will have variation in shooting percentage. On 400 shots the difference between a 7% guy and an 8.5% guy is 6 goals. If I can, even at a 50% confidence level, sign a 7% guy over an 8.5% guy for the same money (if corsi is more or less the same) I’ll do it.

              This is why I prefer to use a goal based analysis. Corsi matters. Shooting % matters. Goal production combines both. Plus goals are not judgmental or result in arena bias. The key is getting a large enough sample size so that the benefits outweigh the small sample size issues. I peg that at around 1 full season, through using multiple seasons will certainly provide far more reliability.

              That is my analytical philosophy, though (right now anyway) I seem to be in the minority.

            • October 8, 2011 at

              Responding below to you, since we can’t keep shrinking comments, David Johnson

      • Vic Ferrari
        October 9, 2011 at

        David

        Off topic, but how is TMCorF% calculated at your site?

        Thanks

    2. October 7, 2011 at

      The “it’ll never be perfect” crowd misses the point completely, which is that from a competitive standpoint, if your understanding of the game is even marginally superior to your competition’s, you can exploit that advantage. Businesses of all sorts make decisions based on incomplete information, but at least with the stats-driven stuff, you can not only take a certain position, you can qualify it with confidence parameters.

      With traditional “I seen it”-based decision making, arguments are often weighed more by the people making them than the merits of the argument itself.

      • October 7, 2011 at

        That’s a great point Dirk. When things are subjective and fuzzy, the influence of authority and convention become stronger.

      • The Other John
        October 8, 2011 at

        Dirk

        I refer to that as an incremental advantage. If I can assemble enough in incremental advantages in my line of work it increases the likelihood of my success. Will love to see how it evolves moving forward because the teams that do not use it or use in grudgingly are going to be surprised how, in a cap world, small mistakes on contracts can hurt a championship calibre roster

    3. JP Nikota
      October 7, 2011 at

      Nice article. I’d really like to read more on the stats used in soccer. I wonder if stats like ‘km covered by a player’ hold just as little value in hockey.

      • Hawerchuk
        October 8, 2011 at

        JP – here’s my soccer site.

        http://epltables.com/

        I haven’t touched it in over a year…I keep meaning to…

    4. Hawerchuk
      October 8, 2011 at

      Tom had this to say: “The type of games that cannot be quantified to the extent of evaluating individuals are the games that battle over territory – Soccer, rugby, football. and hockey.”

      It’s funny that Tom leaves basketball off that list because it seems to be the same kind of game, and yet he agrees that it can be analyzed. I did some analysis maybe six years ago of defensive back performance in football based on a data feed with video tagging someone gave me. You absolutely could tease out individual performance – the vast majority of top pass preventers were pro bowlers. And of course soccer is eminently analyzable.

      I think I would just never use the word “cannot”. I’m sure there are people who said ten years ago that you could never analyze a pitcher’s mechanics statistically because shut up, that’s why, and that a scouting eye was necessary. Pitch F/X blew that out of the water.

      Hockey’s a ways away from a Hockey F/X system, but I’m pretty convinced that a massive database of player and puck movement is something that we could mine to great benefit. Maybe I’m wrong…There’s no such thing as certainty…But I don’t see how you could look at the progression of stats in sports

    5. Hawerchuk
      October 8, 2011 at

      crap. I don’t see how you could look at the progression of stats in sports – constantly disproving the naysayers as most teams begin to integrate it into their decision-making process – and think that there’s some sport out there that it will never penetrate.

    6. Vic Ferrari
      October 8, 2011 at

      I haven’t seen Moneyball yet, it will be interesting to see if it influences MLB further in that direction.

      I’m pretty sure that even now you could have a lot of success as an NHL GM by hiring some of the top sabermetricians, then going opposite George Constanza with their advice. Such is the impact that the Moneyball book had on the industry.

    7. Tom Benjamin
      October 8, 2011 at

      It’s funny that Tom leaves basketball off that list because it seems to be the same kind of game, and yet he agrees that it can be analyzed.

      It is not the same kind of game. It is a game of possession and each possession is the basic unit of analysis. It is very easy to distinguish a game of possession and a game of territory. Basketball teams never voluntarily give up the ball. Hockey (soccer, football, and rugby) are games of territory where teams frequently trade the ball or puck for yards or even inches of ice.

      • Hawerchuk
        October 8, 2011 at

        Tom, I think you’re tying yourself in a bit of a knot to justify the success of basketball analysis while maintaining your belief that hockey cannot be analyzed. Individual play in football and soccer have been well characterized over the years (primarily by watching video and recording a lot of “non-traditional” stats) so that’s a point in the other direction for hockey.

        Genuinely curious, btw – why does it matter to you how the game is analyzed? And why does it make sense for coaches to count scoring chances and use them in player evaluation (which is essentially putting a dollar value on performance) but not for someone like Tyler or Vic to do it?

        • Tom Benjamin
          October 8, 2011 at

          Tom, I think you’re tying yourself in a bit of a knot to justify the success of basketball analysis while maintaining your belief that hockey cannot be analyzed.

          I think you should talk to Dean Oliver. When we talked about it – years ago, mind you – we decided the difference between the sports is that basketball was about possession and the team that controls the ball controls the game. Everything a player can do with a possession is counted. A steal or turnover can be valued. So can shot blocks, defensive and offensive rebounds. These coins add up to dollars and dollars add up to wins. Hockey is not a game of possession. It is a game of territory.

          • Hawerchuk
            October 8, 2011 at

            Well…I have talked to Dean Oliver and Roland Beech. And they’ve discarded this way of analyzing basketball in favor of recording micro-events from video. You’re referencing the best of what Dean and Roland came up with 10 years ago, and they’ve moved on. Roland, in particular, rejects the idea of basic basketball stats capturing value.

        • Tom Benjamin
          October 8, 2011 at

          Genuinely curious, btw – why does it matter to you how the game is analyzed? And why does it make sense for coaches to count scoring chances and use them in player evaluation (which is essentially putting a dollar value on performance) but not for someone like Tyler or Vic to do it?

          I think coaches count scoring chances as a way of evaluating their team. They don’t count them to decide whether Ryan Kesler is better than Jordan Staal or Sean Horcoff or Chris Pronger. I don’t care what you or Tyler or Vic do to evaluate players. If you can make some money at it, that’s great.

          Just don’t expect me to take any of it very seriously. Perhaps more importantly, don’t decide that I am closeminded because I’m not impressed by David Vinnik’s fabulous adjusted Corsi numbers. Bill James changed the way I see baseball. Bill James, along with Kahneman and Tversky changed the way I looked at life (including hockey) in terms of randomness.

          Hockey analytics – so far – has not changed my view of anything.

          • Hawerchuk
            October 8, 2011 at

            No idea who David Vinnik is.

            I think there’s a gap in your thinking. I don’t see how coaches can use scoring chances to evaluate their players and yet at the same time counting scoring chances is not how to evaluate different players. I’m going to leave you alone on this; I think you’ll come around, possibly when some team’s coaching staff reveals that they have a team of chance-counters who’ve gone through five years of tape for the entire league.

      • October 8, 2011 at

        Basketball teams never voluntarily give up the ball. Hockey (soccer, football, and rugby) are games of territory where teams frequently trade the ball or puck for yards or even inches of ice.

        I’m trying, and the only time I can think of an NHL team giving up the puck voluntarily is for a dump-n-chase. And the idea there being that it’s just an indirect pass, your teammate should be able to go get it (though its easier said than done). I don’t think hockey can be characterized as something where you commonly intentionally give the puck away. In football, you protect the ball and don’t want to turn it over any time you can help it, with the only exception being punting, which is less about a “gain” and more about not taking a “loss” by not allowing the other team to get good field position.

        Similarly, while in basketball you never want to give the other team possession, you’re forced to by the existence of a shot clock. In fact, the shot clock is an effort to limit team possession, in the name of increased excitement in the game, similar to the football kickoff (old time football used to commonly allow for 0-0 ties when neither team would kick off, and neither team could score). I would contend that basketball is actually pretty similar to hockey in large part because of the way play moves so quickly up and down the field of play, and that possession is pretty liquid. As such, I think if one lends itself to advanced statistical analysis, then the other is probably possible too.

        • October 8, 2011 at

          And it also bears mentioning that the only voluntary incidence of giving away the ball in football, with punting, has had extensive statistical analysis showing that punting is actually the wrong decision most of the time.

          • Tom Benjamin
            October 8, 2011 at

            I’m trying, and the only time I can think of an NHL team giving up the puck voluntarily is for a dump-n-chase. And the idea there being that it’s just an indirect pass, your teammate should be able to go get it (though its easier said than done). I don’t think hockey can be characterized as something where you commonly intentionally give the puck away.

            Teams do it all the time. The most common incidence is when they chip it out of the defensive zone. Then a few inches is worth the puck. Put it another way:

            Basketball teams want the ball no matter where it is on the court. Hockey teams would rather see the puck in their hands behind the opponent’s net than behind their own with the puck. Wouldn’t you?

            Football is the same way. Would you rather have the football on your own one yard line or give the ball to the opponent on their one yard line?

    8. Tom Benjamin
      October 8, 2011 at

      Interesting post, Tyler. Maybe you are right and with every movement of the puck recorded something could be teased out of the chaos. I’ll believe it when I see it though. Same thing with the soccer data.

      Imagine the Oilers have a great first period in their first game. However we want to describe it, the Oilers carry most of the play, generate most of the chances. The score and all the nifty statistics declare they dominated. After Renney tells them they were great, the Oil come out in the second period and the opposite happens. They are the ones dominated in every respect.

      The question is not, “What happened to change the momentum?” My questions are “What were the Oilers doing right in the first period and wrong in the second?” The stats measure the dominance – the result – but everything short of the attempted goal is a swamp. What do we quantify?

      I see the same thing in soccer. I’m not a big fan, but if I tune into a game I can see who is winning – playing better – halfway through a scoreless tie even though both teams are doing the same things. The losing side players don’t have much space. They move the ball more slowly up the field. Eventually an attack peters out or the ball is booted down the field and possession changes without any real threat. The winning side players start out the same way but the ball moves more quickly, and players aren’t marked quite as closely. Suddenly the ball is launched into an open space, an attacker runs onto it and a dangerous sorty is on.

      What is one team doing right and the other team doing wrong? What are the individual players doing right or wrong? What can we measure if we can’t answer those questions with some specificity? What if the answer is something like team cohesiveness?

    9. October 8, 2011 at

      One thing that will likely help NHL analytics going forward, I’d imagine – we know the NHL has worked on microchipping the puck for purposes of making goal review and such a bit easier. Currently, they apparently aren’t quite there, but if you know anything about technology, you know we’re probably going to get there pretty soon, to have the puck microchipped and able to handle both temperature and the abuse. Once we’re there, its not going to be much more to basically track the pucks entire movement over the game (something we’ve seen experiments with over at BTN), puck speed, contact, etc. As soon as we get that microchip in the puck, the amount of data we’ll have to work with is going to be huge, and largely automatically generated.

    10. Hawerchuk
      October 8, 2011 at

      Tyler

      Moen, Marchant, Pahlsson are all bad on the road too.

      • Hawerchuk
        October 8, 2011 at

        Using G+S+M, I find 56 players with 1500+ on-ice shots over the last four seasons who have an on-ice shooting percentage 2sds above league average on the road and 44 players 2sds below league average.

        There are way more players who are beyond 95% confidence at home than on the road. Some of that’s bad shot-counting, but some of that is last change resulting in getting matched up against weaker opponents who you can finish better against. For the guys with poor finishing ability it doesn’t seem to matter whether they’re at home or on the road.

        • Hawerchuk
          October 8, 2011 at

          Also, as a quick check, Odd-Even R^2 for on-ice shooting percentage is in the 0.25-0.28 range for all shots, and in the 0.15 range for Road and 0.1 for Home.

          So we need to regress these results, even over thousands of shots, pretty heavily. For example, Gaborik:

          Overall on-ice sh%: 8.8% (G+S+M)
          Average sh%: 6.41%
          Regressed 73%: 7.06%
          Over 2502 EV shots: +16 goals above average

          So Gaborik – who’s #1 in the league by a wide margin – is worth on the order of six goals (one win) per season above average. Some of that is his own shooting talent, of course.

          • October 8, 2011 at

            That’s fine math but in reality the impact seems to be much greater. Gaborik’s on ice shooting percentage over the last 4 seasons is 11.16% on 1559 shots. As a result he was on the ice for 174 goals for while a more average 8% guy would have been on the ice for just 125 goals. That’s a gain of 49 goals (+39%), or 12 per year, and that includes 2008-09 in which he played just 17 games. Taking just the 3 other seasons and he has average +14.5 goals/season.

            Crosby and his linemates generate more shots so while Crosby’s on-ice shooting percentage is a little below Gaborik’s, they score on average 16 more goals per season than an average 8% guy.

            Now, either these guys are extremely lucky, or these guys actually can drive shooting percentage to a significant degree. Crosby’s on-ice shooting percentage the past 4 seasons are 11.53%, 11.16%, 10.07%, 11.2%. All well above average and very consistent. He (and/or his linemates) clearly has a unique skill that has a significant impact on scoring goals. Gaborik’s aren’t quite as consistent, but still well above normal every season. You can provide math suggesting they should regress, but in reality they don’t. As a result, any corsi analysis on these guys will greatly under value them.

            Oh, and we haven’t even taken into account the fact that most likely these guys are facing players that are generally good defensively with lower than normal shooting percentages against. It makes their performance even more special.

            • Hawerchuk
              October 9, 2011 at

              David,

              This is even-strength Goals+Shots+Misses (G+S+M), so Gaborik is at 8.8%. Including misses reduces arena bias.

              Keep in mind that *observed* performance is not equal to talent. You need to regress to the mean.

            • October 9, 2011 at

              Maybe I am an idiot and don’t understand the regress to mean concept but if some guy posts an 11% shooting percentage year in and year out it seems to make sense to me that that 11% is probably pretty close to his talent level. Similarly, if I see another guy, lets call him Travis Moen, who consistently has a shooting percentage well under 7% it seems fair to me to believe his actual talent level is somewhere well under 7%.

              As for arena bias, it’s probably relatively minor for most players, but I will point out that Jordan Staal has the same arena bias as Crosby and a 4-year shooting percentage of 8.16%. Thus, my numbers above stand as reasonable. Plus, every time I hear someone cite “arena bias” it just gives me yet another reason to devalue corsi and believe that evaluating goalies purely on their save percentage is flawed.

            • Hawerchuk
              October 9, 2011 at

              “if some guy posts an 11% shooting percentage year in and year out it seems to make sense to me that that 11% is probably pretty close to his talent level.”

              We’ve been through this before. Even if everybody had the same talent level, there would be some guys, who, just by sheer chance, would post very high on-ice shooting percentages over the course of four years.

              The best way to tell if you’re making a claim that makes sense is to take the number of goals that you estimate a talent is worth and translate that first into wins and then into dollars.

              Let’s say that Gaborik is worth +12 goals per season above average (assuming he’s injury prone too) and that NHL replacement level is roughly -12 goals per season below average. That puts Gaborik at +24 goals above average per season in limited playing time and probably at +35 in a full season. That’s roughly six wins or $18M in salary.

              Now some of that is line matching and some of that is his linemates’ talent, but Gaborik is also a + player in terms of directing shots on goal, and he has PP talent too, so your estimate of his talent still puts him in that range. Gaborik makes $7.5M per year…Are you saying he’s paid less than 50% of what he’s worth?

    11. David Staples
      October 8, 2011 at

      The good news is that more than 30 years ago, some smart guy actually came up with the singles, doubles and triples of hockey.

      He did so be marking down all scoring chances and then reviewing them to see which players contributed to the chances for, and deserved a plus mark, and which players made defensive errors on the chances against.

      Since that time, numerous NHL teams have used this system to help them figure out who was playing solid two-way hockey, as Dave King of Phoenix or Jerry Dineen of the Rangers will tell you (at least if Dineed would grant an interview).

      The name of the genius who figured out this is the right event in the game to study, and the right way to study it?

      Roger Neilson.

    12. October 8, 2011 at

      On 400 shots the difference between a 7% guy and an 8.5% guy is 6 goals. If I can, even at a 50% confidence level, sign a 7% guy over an 8.5% guy for the same money (if corsi is more or less the same) I’ll do it.

      This is why I prefer to use a goal based analysis.

      In and of itself, within your example, then yeah, I’d agree with you. Everything else being entirely the same or extremely close, then yeah, the one is probably a marginally better bet. However, you’re talking about two pretty much identical players, and I’d probably argue that such a situation is not that common. Or that if it is, what you really need is to get MORE advanced stats, of other kinds, to help determine the other talents these guys have, besides the ability to shoot the puck. Separating a guy’s performance from his linemates, things like that – the point being that two players probably are not identical, in that they do different things to generate their identical end result. That becomes a bigger thing in terms of trying to figure out if maybe a certain guy’s style of play is going to fit better on your team or in your scheme.

      • October 8, 2011 at

        But, if players can influence shooting percentage (and I argue it does), why should we ignore it (assuming we overcome sample size issues)?

        Or that if it is, what you really need is to get MORE advanced stats, of other kinds, to help determine the other talents these guys have, besides the ability to shoot the puck. Separating a guy’s performance from his linemates, things like that – the point being that two players probably are not identical, in that they do different things to generate their identical end result.

        That’s a fair comment. We should look at quality of teammates and opposition and all that stuff. I am not arguing against that, I am just suggesting there are better options than to do all that using a metric (corsi) that may only tell us half the story.

    13. October 9, 2011 at

      We’ve been through this before. Even if everybody had the same talent level, there would be some guys, who, just by sheer chance, would post very high on-ice shooting percentages over the course of four years.

      Sure, it is theoretically possible and it has probably happened once or twice. The thing is, the names that do do that, the names that are at the top of the shooting percentage list aren’t random. They are really good talented offensive players, often with very good corsi stats too. And the players at the bottom of the list aren’t considered talented offensive players. You can argue that statistically speaking what Crosby and Gaborik has done may possibly be luck, but if it were luck we’d probably see some Travis Moen type names up there too. But, for the most part, we don’t.

      Let’s say that Gaborik is worth +12 goals per season above average (assuming he’s injury prone too) and that NHL replacement level is roughly -12 goals per season below average. That puts Gaborik at +24 goals above average per season in limited playing time and probably at +35 in a full season. That’s roughly six wins or $18M in salary.

      I don’t think you can break it down quite that simply. Unless Gaborik was playing with replacement level players (which he wasn’t) some of that +24 has to be shared among his linemates. Just as we do with corsi, we need to take into account quality of teammates and quality of opposition to be able to put a value on Gaborik. That is not in dispute. The point that I am trying to make, and clearly you disagree which is fine, is that IMO players can drive (and suppress) shooting percentage and thus a corsi based analysis will never tell us the complete story even if we were able to perfectly account for quality of teammates, quality of opposition, luck, and everything else. IMO corsi analysis is starting from an incomplete foundation which is why I don’t generally take that route in player evaluation.

      I think it needs to be said that a lot of shooting percentage is driven by playing style as much as talent. Players, such as Crosby, that are expected and asked to provide offense usually have elevated shooting percentages. They often have elevated shooting percentages against as well. They play a high risk, high reward type game and hope their shooting talent (and to be fair puck control talent) will surpass their oppositions. Players, such as Moen, who are asked to play a defensive role have very poor shooting percentages, but they also often have very low shooting percentages against. They play a low risk, low offensive reward game. Good teams have both types of players on their roster. Score effects on shooting percentage are an example of what happens when players alter playing style (to protect a lead to to get back in a the game with a goal).

      • Hawerchuk
        October 9, 2011 at

        See my response to Tyler’s comment below. You can put some numbers on how much you actually think those guys drive results.

    14. Tyler Dellow
      October 9, 2011 at

      As far as I can tell, Gabe would bet that the 9.5% or better guys there will, collectively, have an s% percentage that is lower than whatever their collective s% is for the past four years in 2011-12. David disagrees.

      Seems the only thing to agree on is the terms.

      • Hawerchuk
        October 9, 2011 at

        And just because bettors should have access to information…I looked at all players who were on the ice for 1000 shots for in 07-08/08-09 and 09-10/10-11. That’s 459 players in total.

        - R^2 of 07-08/08-09 on-ice shooting % to 09-10/10-11 on-ice shooting %: 0.16 (0.15 if misses are included)
        - Number of top 36 players in the first two seasons who were in the top 36 in the second two seasons: Four: Bobby Ryan, Corey Perry, Ryan Getzlaf (hmm…), Brenden Morrow
        - Regression to the mean for the top 36 from those first two seasons: 81%

        David – Tyler has picked out 36 guys who are two standard deviations above the mean over 2007-08 to 2010-11, with an average shooting % of 10.07%. What is your over-under for their on-ice shooting percentage in 2011-12?

        • Hawerchuk
          October 9, 2011 at

          Oh, and christ, Travis Moen just scored on a breakaway to put the Habs up 4-1 over the Jets.

          • Tyler Dellow
            October 9, 2011 at

            Heh. I tweeted at you when this happened. Hilarious.

        • October 9, 2011 at

          For interest sake, lets go with these 10 players:

          Gaborik
          Crosby
          Ryan
          St. Louis
          H. Sedin
          Toews
          Heatley
          Tanguay
          Datsyuk
          Horton

          Those 10 players are from 10 different teams so we won’t be doubling or tripling up several teams. I’ll predict that 8 of those 10 players will end up above 9.5% and as a group they will average around 9.75% or better. I considered including Spezza and Kovalchuk but I think Spezza will be playing with a lot of crappy teammates that will drag him down and I think the Devils style of play inhibits a player from achieving elevated shooting percentages though I would not be surprised if either or both approached or surpassed the 9.5% mark.

          Of course, the outcome of this “bet” is irrelevant in the debate though. Win or lose, I still believe players can drive/suppress shooting percentage.

          • Hawerchuk
            October 9, 2011 at

            The group of forwards you picked = 10.37% on-ice shooting percentage
            Average on-ice shooting percentage for forwards = 9.12%
            Your prediction = 9.75% = 50% regression to the mean

            For 800 on-ice shots, that’s five goals above average. It doesn’t seem like you believe there’s much talent there after all.

            • October 9, 2011 at

              Not sure where you got your numbers, but I get those 10 forwards having a 5v5 on ice shooting percentage of 10.31% (minor difference from your numbers) over the past 4 seasons and a league average on-ice shooting percentage for forwards of 7.94% (significantly different from you). That makes my prediction of 9.75% just under 24% regression to the mean which is sizable, but still keeps those forwards among the top 5% of all forwards.

              But, having taken a closer look at the numbers I’ll predict 10% on-ice 5v5 shooting percentage for those 10 players. Those players averaged 1049 shots while they were on the ice last year so on at 10% shooting percentage it means 21.6 goals above average.

            • Hawerchuk
              October 10, 2011 at

              7.94%? Tell me again what you’re looking at – even-strength only? Just shots on goal and goals? Players with a minimum number of shots?

              The other question…If we assume 25% regression to the mean, that puts replacement level around 6.1%. I think you end up with more goals scored attributed to talent than is possible.

            • October 10, 2011 at

              Take all the goals scored at 5v5 even strength and divide by all the shots taken at 5v5 even strength. This is the league-wide average on-ice shooting percentage.

              Otherwise, take all the players with >2000 minutes of 5v5 ice time over the past 4 seasons. That group of players has an average on-ice 5v5 shooting percentage of 8.096%.

            • Hawerchuk
              October 10, 2011 at

              2007-08 to 2010-11 ESPN EV shooting %, players with 1000+ shots = 9.26%
              2007-08 to 2010-11 ESPN EV shooting % = 8.36%
              2007-08 to 2010-11 NHL EV shooting % = 8.29%
              2007-08 to 2010-11 NHL 5v5 shooting % = 8.27%

              We don’t want to include players with sub-replacement shooting skills in this analysis.

            • October 10, 2011 at

              But if your theory is that all that matters is corsi, why is replacement level even relevant? You can’t argue that shooting percentage isn’t at all significant and then argue that we can’t use sub-replacement level players because they pull down the average shooting percentage. Are you not confident in your players can’t drive or suppress shooting percentage statement?

              Besides, I am using all players who have 2000 minutes of 5v5 ice time and got 8.096%. Players who are sub replacement level don’t get 2000 minutes of ice time.

          • Mike
            October 10, 2011 at

            I’m not entirely sure what your proposal actually proves – that elite players can consistantly shoot above league average and/or drive up shooting percentage?

            That may be true (and probably is), but I’m not sure what it tells us about the other 350 forwards in the league.

    15. Anon
      October 13, 2011 at

      Too bad the traditionalists were so up in arms over Fox-trax (tracks), etc. I saw great use for this in tracking the puck & individual players. Oh well, the NHL is just 15 yrs more behind…

    Leave a Reply

    Your email address will not be published. Required fields are marked *