There’s something unfortunate that happens when people with an interest in sports stats become simultaneously interested in turning a profit from that interest. The desire to generate a profit creates an incentive not to publish the guts of your work, so as to create a body of proprietary knowledge that people will pay you for. This prevents the sort of informal peer review process that has developed from grinding into gear. It also encourages you to market yourself, overstating your confidence and not identifying any weaknesses in your work. The feckless media fuel this, as they lack either the ability or desire to check claims.
Parenthetical: this may well be an ability problem. The Toronto Star’s Damien Cox and the Boston Globe’s Kevin Paul Dupont, while not amongst the most power and influential people in hockey as determined by The Hockey News (smirks), both have awfully large bully pulpits. They both chimed in on Twitter this morning on the NHLPA’s refusal to acquiesce to the NHL’s planned re-alignment.
Cox, in a series of tweets: “After decades of travel across North America, the NHLPA is suddenly concerned about travel. Joke. Huge one. Only bigger joke is belief its harder to make playoffs in an 8-team conference than 7-team. Amazing how some will believe anything. Supposed playoff “unfairness” in proposed alignment is same nonsense as saying going to bigger rink takes out most expensive seats. People, the fight for the fourth playoff spot in any of the new conferences won’t involve the 7th or 8th team. It’s not a factor.”
Dupont: “All play 82 games. Top 4 teams in ea conference make dance. Red herring.”
You would not, I think, need to be particularly gifted in mathematics to realize that teams in a seven team conference have a 4/7 chance at a playoff spot in a given year and teams in an eight team conference have a 4/8 chance at a playoff spot. Cox seems to inexplicably think there will be an equal dispersion of skill levels in the various conferences, with the eight team conferences just having an extra terrible team tacked on. It’s amazing.
Even if division isn’t your bag, you can simply look at the last time that the NHL had a four division setup, with divisional playoffs first and the problem becomes obvious. In 1990-91, Philadelphia finished fifth in the Patrick with 76 points. That would have seen them into the playoffs in every other division. In 1987-88, the Rangers and Isles missed with 82 and 81 points respectively, which would have seen them through in every other division. In 1985-86 and 1986-87, Pittsburgh missed with 76 and 72 points, respectively, totals that would have seen them make it in the Smythe or Norris Divisions.
While you can’t just say that those teams would have made it in any other division, what those seasons and, in particular, 1987-88, show us is that you aren’t guaranteed to divisions of equal strength. The number of teams in your division matters because it reduces the odds that you’ll run into four 100 point teams while you’re the guy with 95 points. This is obvious, but there it is.
There was a study that had a certain segment of the hockey world all heated up last week as Marc Appleby of Powerscout (“…known as the ‘Moneyball’ of the NHL” although by whom is not clear) tweeted out a link to a press release about a study that his company has done. I asked him for a link to something that sets out the research a little more clearly; no luck. This is what we get. Key paragraphs:
The results will surprise many NHL hockey fans because the PowerScout advanced graphs do show that some fights can have a significant impact on a game, even be game-changers in some cases. To evaluate this impact, they used a graph called the Momentum Meter™.
To gain a larger scope of the influence of fights on NHL games, the company analyzed 1,563 fights going back to the 09/10 season using their game archives. They have determined that a fight has a positive effect on at least one team’s momentum in 76% of fights and increases the momentum for both teams about 1 out of every 4 fights (23%). This research was done using the Momentum Meter, looking specifically at the increase in team momentum in the three minutes following the fight.
Appleby agrees that the controversy is far from over and believes that PowerScout’s unique momentum perspective and research will likely add to the controversy as fans can now clearly see how fights impact games.
The paid media got them a pile of free media. I believe one of the principals popped up on Oilers Now. There was a lengthy story on the Associated Press wire. Here’s how that article opens:
This might be the last thing fans and critics on one side of the longest-running debate in hockey want to hear, especially in light of the rising number of concussions:
The first-of-its-kind statistical analysis of the sport confirms the dirty little secret coaches and players have known since the dawn of the NHL. There’s no more readily available, sure-fire way to shift the momentum of a game than to send a player out to start a fight.
I can already hear those guys saying that they didn’t write the story. Fine. But it’s consistent with the tone of their press releases, so they can hardly complain.
It’s almost impossible to evaluate their claims on the basis of what’s been published. As I understand things, what they’re basically saying is that shots increase in the three minutes after a fight relative to the thirty seconds before. This is awfully interesting – as I’ve previously documented (without a PR blitz), there’s a marked decrease in shooting volume as you go from first liners to second liners, second liners to third liners and third liners to fourth liners. We know that the majority of fights involve players who aren’t first liners and probably not second liners – it wouldn’t surprise me to learn that 70 or 80% of hockey fights involved guys who are bottom six, bottom defensive pairing types.
Now, there’s a logic argument here. If we know that most fights involve guys who get less ice time and we know that guys who play less tend to see fewer shots for and against when they are on the ice, we’d expect that shots would increase after a fight because the fight probably involved guys who are less likely to see shots taken by either team when they were on the ice, meaning that it’s more likely that guys who see more shots when they are on the ice will be coming over the boards in the next three minutes.
Is that what these guys have found? I don’t know, because they haven’t presented their data in such a way that their claims can be evaluated. I’d bet some money on that being the case though. Until they show how they’ve controlled for these and various other confounding effects, this is what lawyers call a bald claim.
This sort of thing is all the more aggravating because statistical analysis in hockey is a pretty new field, with lots of people who doubt that it can add anything of value. It’s irresponsible for people interested in this stuff to make sensational claims that aren’t properly documented and able to be reviewed and challenged by others but this stuff is rife in sports analysis – Baseball Prospectus was particularly bad for it. It’s terrible because, in the long run, it leads people to conclude that there’s nothing you can learn from this stuff.
I’ll bet it sells better than an Excel generated graph on a website showing that there are fewer shots when lesser players are on the ice though.